Problems uploading large files to some web sites (long)
- From: Clever Monkey <clvrmnky.invalid@xxxxxxxxxxxxxxxxxxx>
- Date: Mon, 15 Jan 2007 12:08:03 -0500
This may not be a strictly OpenBSD issue, but it involves networking through an OpenBSD edge box, and I've completely run out of ideas.
Network:
NATted OBSD 3.1 running pf with all internal hosts connecting via ethernet through a switch or a wireless access point connected to the switch. OBSD handles all DNS and DHCP. OBSD is the edge box to an always-on ADSL modem connecting via PPPoE (via userland pppd). My ISP gives me a static IP.
I have a Flickr photostream that I've had since before Flickr was bought by Yahoo, and was still in Vancouver. On or about the time the servers were moved to the Yahoo net I became unable to upload photos via their upload form.
Actually, more correctly, I became unable to upload files much bigger than ~8KB. I worked with Flickr for a bit, trading tcpdumps of various trials and never really found a solution. I had changed _nothing_ on my OpenBSD 3.1, so I chalked it up to something weird that Yahoo did or perhaps something stupid my ISP did.
Of course, I could use other photosites just fine, so I switched to another one and never looked back.
Fast-forward 2 years. Suddenly I am unable to upload any file larger than ~8KB with any of their interfaces, including directly using the site API via curl. tcpdump shows my host sending big data packets until the window fills up, and we never get an ACKs back. The connection stalls and times out. If I try a smaller file it just works. I have, again, changed *nothing* on the edge box. It goes months between reboots and months between the timestamp being modified on any file in /etc or /usr.
Things I've tried:
- Using different hosts internally. All fail in the same manner.
- Tried yet another photosite. At least one works fine -- I can upload copious amounts of JPEG data at a time.
- Uploading via email works fine, but it occurs to me that I may not have been using my local SMTP server if I sent from one of my IMAP accounts.
- Tried tweaking (i.e., changing from their default) various sysctl values related to MTU discovery and other inet tweaks that might be related. My sysctl.conf has nothing uncommented other than the setting to allow the box to route packets.
- Disabling pf (actually, swapping in a default pass in/out ruleset). If I do not do this and look at the pf states, you can see 30-40 stateful connections opened (exernal site -> edge box -> internal box running the web session) up while an internal host spins trying to upload stuff. pflog0 has actually never worked for me (via tcpdump), so I can't see what, if anything, is being dropped. (Hence, my test of not filtering anything).
- Trying the same tests from another network (i.e., the office). Works fine, naturally.
- Trying the same tests on my home network, but with my desktop connected via a VPN to the office network. I recall this worked fine, but can't recall if I used a larger file to test with or not.
Things I've yet to try:
- Seeing if the new in-kernel PPPoE on the latest release of OBSD changes anything.
- Seeing if attaching directly to the DSL modem changes anything. I'd like to say it works, but I think I tested only with a small file before I realized that file size was a significant factor.
Upgrading this edge box, duplicating my DDNS, SMTP, DHCP, SMB, httpd and RDBMS setup, all the user accounts is such a pain I'd rather stick forks in my eyes. I can probably just try the basic connection stuff, but even that will require me figuring out DNS/DHCP _again_, and I'm one of those people who think that sendmail.cf files are easier to maintain that DDNS setups. Grump.
I'll do my best to get that sorted, but I'm looking for ways to narrow down what the problem might be. I'm probably not using the tools at my disposal very well, and admit that the interaction between all those TCP options, PPP and the greater unwashed ISP implementations makes little sense to me.
All I know is that it used to work, and now it doesn't. The part I don't like is that things failed like this before.
.
- Follow-Ups:
- Re: Problems uploading large files to some web sites (long)
- From: Clever Monkey
- Re: Problems uploading large files to some web sites (long)
- Prev by Date: Re: Slow data transfers across LAN
- Next by Date: Soekris 4801+VPN1411+OpenBSD 4.0 = Corrupted MAC on Input
- Previous by thread: Another mp3 file play problem
- Next by thread: Re: Problems uploading large files to some web sites (long)
- Index(es):
Relevant Pages
|
