Re: Using shell commands versus C equivalents

On Wed, 13 Jun 2007, Rick C. Petty wrote:

On Wed, Jun 13, 2007 at 10:23:36AM -0700, youshi10@xxxxxxxxxxxxxxxx wrote:
On Wed, 13 Jun 2007, Joerg Sonnenberger wrote:

Should I briefly lock (flock) the file when running open/fstat/fchmod then
to avoid issues? This may become a problem as pkg_*/make becomes more
parallelized (another student's goals for his SoC project).

I wouldn't bother. What issue are you trying to avoid? If one process is
trying to chmod +x and another is trying to do a chmod -x, it shouldn't
matter if you lock between the fstat/fchmod-- someone is going to win
anyway. This operation is not something that needs to be thread-safe.

Needless to say, pkg_* is by no means threadsafe in its current form
though. It uses some global vars that are currently not mutex locked, and
this type of file access is another issue (I wonder if spinlocking or
sleeping waiting for flock to finish would be better in this case).

Does pkg_* use multiple threads? I was under the impression each pkg tool
used a single thread (i.e. no threads) to do its operations and that they
wait for system(2)-type calls as needed. Maybe I'm not clear by what you
mean when you say "global vars".

Well, I mean that as it currently stands there are several variables used globally for setting attributes per package (I'm not at my machine right now so I can't look them up until ~6pm PST).

Now another question is whether the pkg_* tools can handle multiple
processes managing the ports at the same time. For the mostpart, this is
true. Without looking at the code, I would expect that the only
contentions would be when trying to update the +REQUIRED_BY files.
Everything else should be just fine; you're not supposed to be installing
the same port multiple times at the exact same time, but maybe a lock could
be held on the package directory (i.e. /var/db/pkg/$PKG_NAME). Again, I
don't believe this is strictly necessary.

Currently, no, and this is a condition that's contingent for a fellow SoC'er's project. The mentor said that all that *should* occur is there should be an flock, but that was it. So instead of making more work for him and since I am modifying pkg_* already, I thought it would be best to just make my modifications to simplify his end (he still has a ways to go on the dependency tracking I think).

It goes a bit deeper than the +REQUIRED_BY files, in particular with the +CONTENTS, etc files as the pkg_* tools are enumerating the packages currently on the system, their dependencies, owning files, etc. Perhaps a global .lock file of some kind in the package directories would be the way to go though.

Thanks,
-Garrett

_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: I cant flock, it returns 0, any ideas.
    ... > I can't flock, it returns 0, any ideas. ... You can only hold a lock on an open file. ... will fail but attempts to get another LOCK_SH lock will ... this is not how you pause the script! ...
    (comp.lang.perl.misc)
  • Re: I cant flock, it returns 0, any ideas.
    ... # returns zero when I try to flock ... >> I wanted to run this script to see if it would affect my other perl ... If someone holds a LOCK_EX lock, all attempts to get a lock will ... > the flock call will fail, and if you don't flock will just not ...
    (comp.lang.perl.misc)
  • Re: UW imap-2006b: 64 bit problem
    ... UW imapd is obliged> to do considerable more work on SVR4 systems than it does on BSD and> Linux systems which offer flock() locking as an alternative to POSIX> locking. ... IEEE Std 1003.1-1988 requires that all fcntl() locks associated with a file for a given process are removed when *any* file descriptor for that file is closed by that process. ... Put another way, before any library routine opens a file, it must be aware of what files the application and any other libraries have open and locked; otherwise the library routine will remove the lock unexpectedly when it closes the file. ...
    (comp.mail.imap)
  • flock() replacement?
    ... lock/unlock library code. ... It uses flock() to synchronize ... pthread mutex lock. ... I have tried POSIX sem lock, ...
    (comp.unix.programmer)
  • Re: flock() replacement?
    ... lock/unlock library code. ... It uses flock() to synchronize ... pthread mutex lock. ... linux 2.6, flockconsumes the most user and sys times. ...
    (comp.unix.programmer)