Re: Cleaning up FILE in stdio..

On Wed, Feb 27, 2008 at 11:38:33AM -0500, John Baldwin wrote:
You could change _file from 'short' to 'unsigned short' without breaking
the ABI - this would allow either 65535 or 65536 file descriptors (I'm
not sure whether _file == -1 is special or not). This would postpone
the problem for some time.

-1 is used a lot in the stdio code for file's not backed by an fd. My problem
though is that this doesn't help with existing binaries that are already
compiled (which is what I have to deal with). Had fileno() not been inlined
I would have been ok, but that's pretty much done for me as far as my current
problem on 6.x. Had I just been able to change FILE * and not had inlines,
then a new fopen would have worked fine in my case.

My suggestion was based on short and ushort having the same size and
(short)-1 and (ushort)65535 having the same bit pattern. Any code
accessing fileno() should not be checking or validating the result
but just passing it to low-level I/O routines. This would provide the
following:
Existing code New code
short ushort
FD sign-extended zero-extended
-1 -1 65535
0..32767 0..32767 0..32767
32768..65534 -32768..-2 [*] 32768..65534
65534 EMFILE EMFILE

[*] This could potentially be fixed using libc or kernel shims.

e) Don asbestos underwear and re-arrange struct __sFILE to grow _file etc.

We can't do e) because thanks to symbol versioning, 8.x and 9.x will have
libc.so.7, so a 7.0 binary will still use the brand new libc, so it has to
preserve the ABI of the currently exported fields pretty much forever.

Erk. I forgot about that.

think we can get away with renaming '_file' to '_ofile' and adding a new 'int
_file' at the bottom of the struct and making sure '_ofile' is always in sync
(when possible, truncated when _file is too bug).

Truncation opens up the possibility that old executables could fopen()
lots of files (without getting any indication of a problem) and then
use fileno() to reference a truncated _ofile - causing it to access
some totally unrelated file. Admittedly, that is no worse than would
happen today.

Also, I think we can do the new _file in HEAD for 8.0 w/o any worries. I
don't think waiting until 9.0 buys anything there.

I was thinking in terms of changing _file to int without backward
compatibility. I'm not sure that that could be done for 8.0 (though,
as you have pointed out, it can't be done at all).

Given that, I think I'd
rather just patch the current stable branches to handle the edge case better
and work on making _file an int in HEAD (with the ABI compat _ofile).

The EMFILE patch is definitely a good interim step and I support your
efforts in removing the limit on the number of open files.

--
Peter Jeremy
Please excuse any delays as the result of my ISP's inability to implement
an MTA that is either RFC2821-compliant or matches their claimed behaviour.

Attachment: pgpfWbp5VFlN2.pgp
Description: PGP signature