Re: large file support && ! large file support

On Sat, 19 May 2007 19:18:24 -0700 Frank Cusack <fcusack@xxxxxxxxxxx> wrote:
| On 20 May 2007 01:53:19 GMT phil-news-nospam@xxxxxxxx wrote:
|> | If your library is an interposer, you don't need to do anything
|> | special at all though. 32-bit and 64-bit (LFS) library calls are
|> | independent from each other, with the appropriate call invoked via
|> | macros at compile time when building with LFS support. So you just
|> | need 2 versions of each function you are interposing on.
|>
|> That's what I want to avoid. I don't want to create a "permanent transition"
|> like LFS has become. I don't want two different interfaces.
|
| That's not possible, if backwards compatibility is to be retained.

Let me clarify. There are two different ways to make reference to having
more than one interface. Clearly if someone compiles my library on a
64-bit machine, that is a truly different interface at the ABI level than
one compiled on a 32-bit machine. Similarly, at the ABI level, interfaces
would be different for different endianness, as well as padding/alignment
issues where structs could be involved, and possibly even the argument
list itself.

The above is one kind of "two different interfaces". The other is having
what might be better termed "overloaded interface" where for each part of
the interface (a function, a struct, a data type, etc), or some subset of
those parts, a corresponding variant definition is created that only makes
sense, or is only needed on, certain categories of architectures. What I
mean here are things like the added "64" functions in LFS, new data types
with "64" in them, and various other things. On a 32 bit machine these are
different. On a 64 bit machine they are pointless.

The kind of thing I want to avoid is the latter.

I see nothing wrong with having a "32-bit pointer / 64-bit file reference"
type of architecture that provides compatibility for legacy programs with
an extra legacy library that provides the ABI for older programs that expect
"32-bit pointer / 32-bit file reference". They could also make an alternate
toolchain to compile programs in the legacy architecture (for example the
legacy library itself which can still need maintenance). But the default
should be the forward architecture.

A very small amount of the POSIX standard failed to have the ability to
"just recompile" to get correctly written programs working on the new
"32-bit pointer / 64-bit file reference" architecture (e.g. why we had
to have fseeko and ftello because of the lack of ODTs). Programs using
fseeko and ftello in lieu of fseek and ftell should be able to simply be
recompiled for the default architecture.

Too many programs would have to be fixed. So someone decided rather than
recode those programs that were broken, we'd have to recode all programs
to add large file support in the transition. So in an effort to avoid
more work, we have a lot more work. It was a bad idea from the get-go.
Someone called it a transition. Looks like it will be around for many
years, maybe decades.


|> I want to make
|> things work as they should, where the usage of an interface involves using
|> the correct ADT which is defined to the correct size and time. With the
|> exception of the fseek and ftell functions, which got replaced by fseeko
|> and ftello, the whole standard interface _should_ have been done by having
|> simply redefined all the data types. There should never have been "64"
|> versions of any syscalls, library functions, data types, structs, etc.
|> They ended up doing this for various reasons related to legacy programs.
|
| I disagree strongly; it was done the correct way. Supporting
| so-called legacy programs is absolutely essential.

This depends on the definition of support. If you want original binary
programs that source is unavailable for to also work with large files,
then that is quite a stretch. But adding the "64" extensions is in no way
a form of supporting legacy programs, either with or without source.
With two different ABIs, you have legacy support. No need to bloat the
API to do so. You might have 2 library binaries around, and intimate
libraries like libc might well have to be coded to understand which it
is being compiled for (because the kernel might only have the LFS KABI).


| That's not to say that the purist in me doesn't appreciate your position.

But it's more than just a purist issue. It's also a practical issue.
Why force recoding all programs when it would have only been necessary
to recode the broken subset (which I am not convinced is so large).


|> My library has no legacy so I don't want to create any such mess for it.
|> So I'm just trying to understand the specifics of the existant mess to see
|> how I might be able to accomplish this (exactly one way to access each
|> element of the interface, and yet have it always work no matter how the
|> calling programs are accessing the POSIX interface).
|
| I can't imagine how this could be done.

This is my quest.

At this point it seems the way I will do so is to have my API be always
of the "implicit large file" type when either implicit or explicit large
files are used by calling programs. A calling program using implicit
would not really see a difference. A calling program using explicit is
going to have to take notice and deal with API elements with great care.
And those in my library would be the large file type but without the large
file names. It's the ability to be able to call both that will be lost.
So a program might be able to call both lstat() and lstat64() in libc to
reach the kernel in two different ways (and libc may well be translating
lstat() to a 64-bit underlying KABI where lstat64() would be a straight
call). But it would only have the 64-bit API in my library. A program
being compiled entirely without LFS would get the non-LFS API, and have
to link with a non-LFS binary (assuming one gets compiled).

I will have to decide if I want to make the default make configuration
create both 32 bit and 64 bit library binaries, or just the 64 bit.

One thing I think I need to detect on a given platform is if LFS was a
choice at all. If the machine is a true 64-bit machine, that might not
be entirely the obvious case, since there may still be 32-bit program
emulation going on, with support for a 32-bit pointer ABI which could
have either 32-bit or 64-bit file access data sizes, or both.

But I definitely do NOT want to see "64" variations in an API on a true
64-bit machine for programs compiled to the full architecture. Yet with
the POSIX LFS way, we apparently will be stuck with that. It's not so
much a transition as it is a trap.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / spamtrap-2007-05-20-0813@xxxxxxxx |
|------------------------------------/-------------------------------------|
.

Relevant Pages

  • Re: How to "import" an interface into a module w/o `include?
    ... libs but not one file per lib. ... do you mean that you have used 'vlog' to compile your source files ... into ModelSim libraries of those names? ... could possibly know it's an interface). ...
    (comp.lang.verilog)
  • Re: Variable uses a type not supported in Visual Basic (Error 458)
    ... > Microsoft Visual Basic for Applications Software Development Kit ... > interface in a Visual Basic class, ... > generate libraries that use a Visual Basic- or Visual Basic for ... > Rebuild your type library by pressing CTRL+F7 or selecting Compile ...
    (microsoft.public.vb.bugs)
  • Re: How to "import" an interface into a module w/o `include?
    ... libs but not one file per lib. ... do you mean that you have used 'vlog' to compile your source files ... into ModelSim libraries of those names? ... could possibly know it's an interface). ...
    (comp.lang.verilog)
  • Re: Recommendations for JavaScript drop-down menu code
    ... to not having a long enough history working with browsers to ... Obviously these individuals are not aware that there code has these issues following from their inexperience, but now your project manager cannot solve his problem by just using a library unless he has someone who knows the issues themselves and so can point out which libraries properly handle them and which doe not. ... interface and a base implementation of that interface. ... but it is still important that the interface design stage takes into account the need to accommodate the whole range of possible implementations of the interface. ...
    (comp.lang.javascript)
  • Re: TrueCrypt 4.0 Out
    ... >> Again you show that you know very little about information security. ... > If you don't understand the code enough to change a calling interface ... reasons for not making changes to this code unless this cannot be avoided. ... I can certainly now see why you mistkenly think that libraries are ...
    (sci.crypt)