Re: Fwd: Re: pf: BAD state happens often with portsnap fetch update

On Tue, Dec 26, 2006 at 11:50:19AM -0500, Mike Silbersack wrote:

> The random port allocation, because it is completely random, runs into the
> birthday problem if it tries to allocate too many ports: Within a few
> hundred
> port allocations, there's almost certainly going to be a collision. To
> get
> around this problem, the port allocator watches how many ports are being
> allocated, and switches to sequential allocations if it thinks that the
> rate
> of port allocation is likely to result in collisions occurring.
>
> Unfortunately, this switch isn't occurring quickly enough to avoid
> problems;
> I'm not sure if this can be easily fixed (except via the workaround of
> turning
> off randomized port allocations), but maybe Mike Silbersack (CCed) will
> have
> some ideas.
>
> Colin Percival

Colin's description is accurate, but I haven't read up to this point in
the thread, and I need more information.

To prove whether or not this is really port randomization's fault for
using ports excessively quickly (say, within 1ms) or whether something is
going wrong due to ports being used relatively quickly (say, within 1
seconds), please do the following:

1. Disable randomization
2. Set the ephemeral port range to something small like 49152 to 49352.
3. Re-run the test in question.

Tell me how it goes.

Mike "Silby" Silbersack

After about 13 seconds of active fetching, portsnap cycles sequentially
through the remainder of the available ephermal range set as above (200
ports) and it goes ahead and tries to reuse 49152 as soon as it got done
using 49352. tcpdump shows the client host sending SYNs to the squid server
periodically for about 56 seconds, until pf allows it through and a response
completes. A few more ports are allowed through somewhat rapidly, then
at times there are additional waits while the new connections bump up
against pf's enforced timeouts. I let portsnap go on to at least 2600 ports
and it seems to be plugging along slowly and probably would have worked
to completion. I still have the tcpdump capture, and the pf logfile, should
I post them for evaluation, or look at them for something specific, or run
more tests? I'm on the road for the holidays but I can probably check my
email tonight and/or tomorrow.

portsnap shows almost 200 ports downloaded, a pause, continue to download
another 200, pause, etc. Example:

Fetching 8881
patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150....160....170....180....190[wait
here]....200....210....220....230....240....250....260....270....280....290....300....310....320....330....340....350....360....370....380....390.[wait
here]...400....410.(etc)

_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"
_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Fwd: Re: pf: BAD state happens often with portsnap fetch update
    ... Fetching snapshot tag from portsnap2.FreeBSD.org... ... I have to run portsnap several times until it succeeds. ... starts re-using ports from previous connections, ...
    (freebsd-current)
  • Re: [OT] CVSUP (was "Re: Was: Re: Why This Infinite Loop??")
    ... if he is not familiar with the FBSD ports system. ... supposedly an advantage of portsnap. ... The protocol uses no encryption or signing, and any attacker who can intercept the connection can insert arbitrary data into the tree you are updating. ... this means that anyone who can compromise a CVSup mirror can feed arbitrary data to the people who are using that mirror. ...
    (freebsd-questions)
  • Re: gemeral questions (noobish)
    ... yes and no:P I like portsnap. ... For my ports, ... sudo portsnap fetch && sudo portsnap update ... That results on a daily e-mail from cron that looks like this: ...
    (freebsd-questions)
  • Re: Why does portsdb -Uu run so long?
    ... and the other one for ports. ... portsnap, regardless of how fast it is. ... There's no reason to do a buildworld sequence everytime you upgrade the ports ... upgrading with portugrade -arR, I'll be done and using the system. ...
    (freebsd-questions)
  • Re: cvsup vs. portsnap (was Re: cvsup problem)
    ... Each time you run CVSup, it transmits a list of all the files in the ... if your ports tree is almost up-to-date already, ... does not occur with portsnap to any significant extent; ... > why a normal user (eg one that doesn't need to fetch a version of ports ...
    (freebsd-questions)