Re: NGROUPS_MAX in CURRENT - Where are those NFS problems?
From: Robert Watson (rwatson_at_FreeBSD.org)
Date: Tue, 31 May 2005 19:53:25 +0100 (BST) To: Andre Guibert de Bruet <firstname.lastname@example.org>
On Tue, 31 May 2005, Andre Guibert de Bruet wrote:
> This question gets asked over and over on the various lists [*], but I
> have yet to see an answer other than "We are not sure, and it is
> probably not worth it" or "don't hike it too high, performance
> degradation might occur":
> "Why doesn't CURRENT have NGROUPS_MAX set to 32 or 64?"
> The issue with xucred makes sense for the stable branches, but I am not
> sure that the same compat guarantees are provided for current. I have
> used local diffs to 64 for a number of years and have yet to run into
> I have read about fireworks with NFS, and I guess I am confuzed by why
> this would would be the case. My "normal use" of NFS on system with
> users in many groups never seems to have turned up any problems. The
> following was done on an NFS client system with NGROUPS set to 64 and a
> server with 16 in an NFS-mounted directory.
I've had a very specific request to investigate larger values of
NGROUPS_MAX, on the order of hundreds and thousands of groups, and have
started to do so. Unfortunately, I have a lot of other stuff on my plate
right now, so haven't gotten too far. The issues I'm aware of are as
(1) If you need a large potential increase in maximum groups, you need to
move to a variable-length credential structure so you don't waste lots
of kernel memory. I have an initial prototype of this in some branch
in Perforce. Basically, if you ask for <16 groups, you just get
groups in ucred, otherwise you get an extended storage allocation that
contains a ucred. Among other things, you now have to also maintain
an array size. I had hoped to get this into 6.x, but most likely it
will go into 7.x and maybe get backported.
(2) Some kernel components are aware of, and export, group sets. The two
specific ones I'm aware of are the ones you've already highlighted:
the NFS client, and xucred. Additional credentials support was
recently added to UNIX domain sockets, so they likely need more
attention as well. It's not clear to me what the right approach for
NFS is, so we should look at what other people have done -- they most
likely truncate the list during export. It may be we already truncate
in the NFS code correctly, but to be honest, I've not looked. The
problem, btw, will turn up when you try to access something using an
extended group beyond the maximum supported by NFS, and because the
client truncates the group set, access is denied (or granted). I
don't know if any vendors have NFS extensions to deal with this, but
some googling is probably called for.
(3) We need to check use of group sets in user space -- particularly, for
stack-allocated group storage, the group database code, YP, NSS, etc.
I've not yet begun this, but suspect that if you start to support
thousands of groups, you run into problems here. Probably not hard to
fix, but someone needs to do it.
If you're aware of any specific issues, please file PRs for them and point
me at them.
Robert N M Watson
> %id; echo "test" > testing; sync; ls -l testing; cat testing;
> uid=1028(andy) gid=101(users) groups=101(users), 0(wheel), 102(src),
> 106(crevil), 108(nmap), 109(osinc), 110(toast), 112(fpjet), 114(rose),
> 115(mysql), 116(khendon), 117(ffchat), 120(cdcas), 121(masus), 122(stukings),
> 123(uppersixth), 124(brits), 125(ncom), 126(slems), 128(architects),
> 129(parlane), 132(bounce), 133(cryogen), 134(bikes), 135(enterprise),
> 136(sort), 137(bcomp), 138(btestsite), 140(openj), 141(mmouse), 142(ctbm),
> -rw-r--r-- 1 andy wheel 5 May 25 20:17 testing
> As you can see, the output is as one would expect. I would love to provide
> patches / testing on this "issue" but I can't seem to reproduce it.
> [*]: References:
> /* Andre Guibert de Bruet * 6f43 6564 7020 656f 2e74 4220 7469 6a20 */
> /* Code poet / Sysadmin * 636f 656b 2e79 5320 7379 6461 696d 2e6e */
> /* GSM: +1 734 846 8758 * 5520 494e 2058 6c73 7565 6874 002e 0000 */
> /* WWW: siliconlandmark.com * Tormenting bytes since 1980. */
> email@example.com mailing list
> To unsubscribe, send any mail to "firstname.lastname@example.org"
email@example.com mailing list
To unsubscribe, send any mail to "firstname.lastname@example.org"