Re: IPSec tcp session stalling

From: Volker (volker_at_vwsoft.com)
Date: 10/23/05

  • Next message: Jayton Garnett: "Re: Problem with firewall" 
    Date: Sun, 23 Oct 2005 01:56:21 +0100
To: Max Laier <max@love2party.net>

    Max & Co:

    I've just seen I'm using kernel config 'options IPSEC' on both machines.
    Should I try 'options FAST_IPSEC'? Would take some hours for kernel
    recompile. Does the code IPSEC / FAST_IPSEC make a difference (even
    while having not hardware crypto accelerator)?

    May I use FAST_IPSEC even without any hw-crypto devices? While reading
    `man fast_ipsec' I would think it depends on a hw-crypto device...

    Please tell me if we should check IPSEC / FAST_IPSEC and I'll start a
    recompile.

    Volker

    On 2005-10-23 00:40, Max Laier wrote:
    > To try something else: Could you guys try to disable SACK on the machines
    > involved? I haven't looked at the dumps as of yet, but that's one simple
    > test that might help to identify the problem.
    >
    > sysctl net.inet.tcp.sack.enable=0
    >
    > On Sunday 23 October 2005 02:23, Volker wrote:
    >
    >>Michael,
    >>
    >>I not that sure if I'm right in checking what you suggested but when
    >>trying to do ping hostB from hostA with oversized packets through the
    >>IPSec tunnel by:
    >>
    >># ping -c 10 -s 12000 10.128.6.1
    >>
    >>I'm getting replies easily.
    >>
    >>While doing that and tcpdump'ing the gif interface, I'm seeing the
    >>fragmented packets coming in properly.
    >>
    >>If that's a reliable check for MTU than the problem should not be MTU
    >>related. Is there any other way to check MTU problems by using `ping'?
    >>
    >>Thanks,
    >>
    >>Volker
    >>
    >>On 2005-10-22 20:16, Michael VInce wrote:
    >>
    >>>Try sending different sized pings or other packet size control utils to
    >>>really make sure its not MTU related.
    >>>Maybe there is an upstream router thats blocking ICMP fragment packets,
    >>>have you ever seen them? try forcing the creation of some.
    >>>
    >>>Mike
    >>>
    >>>Volker wrote:
    >>>
    >>>>Still having the same problem with an IPSec tunnel between FreeBSD 5.4R
    >>>>hosts.
    >>>>
    >>>>Problem description:
    >>>>scp session tries to transfer a large file through an IPSec tunnel. The
    >>>>file is being transmitted but scp says 'stalled' after 56K (49152 bytes
    >>>>file size). The IPSec tunnel itself is still up even after the scp
    >>>>abort. Other tcp sessions break, too when sending too much traffic
    >>>>through the tunnel.
    >>>>
    >>>>I've taken a closer look to it and tried to get something useful out of
    >>>>the tcpdump but I'm unable to see any errors or I'm misinterpreting
    >>>>something.
    >>>>
    >>>>The connection looks like:
    >>>>
    >>>>extIP: A.B.C.D
    >>>>extIP: E.F.G.H
    >>>>host A ------------------ (internet) ------------------ host B
    >>>>tunnelIP: 10.128.1.6 tunnelIP:
    >>>>10.128.6.1
    >>>>
    >>>>host A just has an external interface (em1) connected to a leased line
    >>>>with a fixed IP address (IP-addr A.B.C.D).
    >>>>host B has an S-DSL connection at xl0, PPPoE at ng0 (IP-addr. E.F.G.H).
    >>>>
    >>>>Both hosts are using gif for the IPSec tunnel.
    >>>>
    >>>>The routing tables (netstat -rnWf inet) are looking good and IMHO the
    >>>>MTU is fine.
    >>>>
    >>>>host A:
    >>>>em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    >>>> options=b<RXCSUM,TXCSUM,VLAN_MTU>
    >>>> inet A.B.C.D netmask 0xfffffff8 broadcast A.B.C.z
    >>>> ether 00:c0:9f:46:ec:c7
    >>>> media: Ethernet autoselect (100baseTX <full-duplex>)
    >>>> status: active
    >>>>gif6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
    >>>> tunnel inet A.B.C.D --> E.F.G.H
    >>>> inet 10.128.1.6 --> 10.128.6.1 netmask 0xffffffff
    >>>> inet6 fe80::2c0:9fff:fe46:ecc6%gif6 prefixlen 64 scopeid 0x4
    >>>>
    >>>>Routing tables (shortened)
    >>>>Destination Gateway Flags Refs Use Mtu
    >>>>Netif Expire
    >>>>default A.B.C.x UGS 2 516686 1500 em1
    >>>>10.128.1.6 127.0.0.1 UH 0 14
    >>>>16384 lo0
    >>>>10.128.6.1 10.128.1.6 UH 0 6017
    >>>>1280 gif6
    >>>>127.0.0.1 127.0.0.1 UH 0 31633
    >>>>16384 lo0
    >>>>A.B.C.x/29 link#2 UC 0 0 1500 em1
    >>>>A.B.C.D 00:c0:9f:46:ec:c7 UHLW 0 112 1500 lo0
    >>>>
    >>>>On host B the interfaces and routing tables are looking like:
    >>>>xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    >>>> options=8<VLAN_MTU>
    >>>> inet 0.0.0.0 netmask 0xff000000 broadcast 0.255.255.255
    >>>> inet6 fe80::260:8ff:fe6c:e73c%xl0 prefixlen 64 scopeid 0x1
    >>>> ether 00:60:08:6c:e7:3c
    >>>> media: Ethernet 10baseT/UTP (10baseT/UTP <half-duplex>)
    >>>> status: active
    >>>>gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
    >>>> tunnel inet E.F.G.H --> A.B.C.D
    >>>> inet6 fe80::260:8ff:fe6c:e73c%gif1 prefixlen 64 scopeid 0x4
    >>>> inet 10.128.6.1 --> 10.128.1.6 netmask 0xffffffff
    >>>>ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> mtu 1456
    >>>> inet E.F.G.H --> 217.5.98.186 netmask 0xffffffff
    >>>>
    >>>>Routing tables (shortened)
    >>>>Destination Gateway Flags Refs Use Mtu
    >>>>Netif Expire
    >>>>0 link#1 UC 0 0 1500
    >>>>xl0 =>
    >>>>default 217.5.98.186 UGS 1 38474
    >>>>1456 ng0
    >>>>10.128.1.6 10.128.6.1 UH 4 2196
    >>>>1280 gif1
    >>>>127.0.0.1 127.0.0.1 UH 0 80424
    >>>>16384 lo0
    >>>>217.5.98.186 E.F.G.H UH 1 0 1456 ng0
    >>>>E.F.G.H lo0 UHS 0 0 16384 lo0
    >>>>
    >>>>While trying to fetch a file by scp on host A (receiver) from host B
    >>>>(sender), I captured the following tcpdump on host B:
    >>>>
    >>>>tcpdump -netttvvi gif1:
    >>>>
    >>>>>000023 AF 2 1280: IP (tos 0x8, ttl 64, id 13202, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>43864:45092(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567
    >>>>>565002838>
    >>>>>000207 AF 2 1280: IP (tos 0x8, ttl 64, id 52187, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567
    >>>>>565002838>
    >>>>>000220 AF 2 1280: IP (tos 0x8, ttl 64, id 33774, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 481770568
    >>>>>565002838>
    >>>>>003524 AF 2 52: IP (tos 0x8, ttl 64, id 42063, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 38952 win 33156 <nop,nop,timestamp 565002844
    >>>>>481770524> 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 48541, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 481770571
    >>>>>565002844>
    >>>>>011203 AF 2 52: IP (tos 0x8, ttl 64, id 60517, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 41408 win 32542 <nop,nop,timestamp 565002855
    >>>>>481770530> 000058 AF 2 1280: IP (tos 0x8, ttl 64, id 15798, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 481770582
    >>>>>565002855>
    >>>>>000246 AF 2 1280: IP (tos 0x8, ttl 64, id 31721, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 481770583
    >>>>>565002855>
    >>>>>005147 AF 2 52: IP (tos 0x8, ttl 64, id 22347, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 42636 win 33156 <nop,nop,timestamp 565002861
    >>>>>481770542> 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 61057, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 481770588
    >>>>>565002861>
    >>>>>020769 AF 2 52: IP (tos 0x8, ttl 64, id 27692, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 565002881
    >>>>>481770547> 000027 AF 2 1280: IP (tos 0x8, ttl 64, id 64167, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609
    >>>>>565002881>
    >>>>>000209 AF 2 1280: IP (tos 0x8, ttl 64, id 45457, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609
    >>>>>565002881>
    >>>>>005260 AF 2 52: IP (tos 0x8, ttl 64, id 53832, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 565002887
    >>>>>481770567> 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 3515, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 481770614
    >>>>>565002887>
    >>>>>011020 AF 2 52: IP (tos 0x8, ttl 64, id 11608, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 565002898
    >>>>>481770568> 000026 AF 2 1280: IP (tos 0x8, ttl 64, id 5848, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625
    >>>>>565002898>
    >>>>>000211 AF 2 1280: IP (tos 0x8, ttl 64, id 39892, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625
    >>>>>565002898>
    >>>>>005641 AF 2 52: IP (tos 0x8, ttl 64, id 7943, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 565002904
    >>>>>481770582> 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8678, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 481770631
    >>>>>565002904>
    >>>>>011072 AF 2 52: IP (tos 0x8, ttl 64, id 38257, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 565002915
    >>>>>481770583> 000025 AF 2 1280: IP (tos 0x8, ttl 64, id 12255, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642
    >>>>>565002915>
    >>>>>000209 AF 2 1280: IP (tos 0x8, ttl 64, id 46257, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642
    >>>>>565002915>
    >>>>>000222 AF 2 1280: IP (tos 0x8, ttl 64, id 4093, offset 0, flags
    >>>>>[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 481770643
    >>>>>565002915>
    >>>>>007065 AF 2 52: IP (tos 0x8, ttl 64, id 18720, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 565002922
    >>>>>481770609> 000025 AF 2 1280: IP (tos 0x8, ttl 64, id 38378, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 481770650
    >>>>>565002922>
    >>>>>011034 AF 2 52: IP (tos 0x8, ttl 64, id 18718, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 565002934
    >>>>>481770609> 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8148, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
    >>>>>64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 481770661
    >>>>>565002934>
    >>>>>005991 AF 2 52: IP (tos 0x8, ttl 64, id 62285, offset 0, flags
    >>>>>[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
    >>>>>1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 565002939
    >>>>>481770625> 010726 AF 2 52: IP (tos 0x8, ttl 64, id 1549, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
    >>>>>ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 565002950
    >>>>>481770625> 005670 AF 2 52: IP (tos 0x8, ttl 64, id 61504, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
    >>>>>ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 565002956
    >>>>>481770642> 011260 AF 2 52: IP (tos 0x8, ttl 64, id 32633, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
    >>>>>ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 565002967
    >>>>>481770642> 005510 AF 2 52: IP (tos 0x8, ttl 64, id 54614, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
    >>>>>ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 565002973
    >>>>>481770650> 104909 AF 2 52: IP (tos 0x8, ttl 64, id 50471, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
    >>>>>ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 565003078
    >>>>>481770661>
    >>>>
    >>>>tcpdump -netttvvi ng0 host A.B.C.D:
    >>>>
    >>>>>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 25895, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10b)
    >>>>>011042 AF 2 128: IP (tos 0x8, ttl 61, id 5786, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf0)
    >>>>>000226 AF 2 1352: IP (tos 0x8, ttl 64, id 36701, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10c)
    >>>>>000216 AF 2 1352: IP (tos 0x8, ttl 64, id 8789, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10d)
    >>>>>004853 AF 2 128: IP (tos 0x8, ttl 61, id 17128, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf1)
    >>>>>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 34888, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10e)
    >>>>>018747 AF 2 128: IP (tos 0x8, ttl 61, id 14828, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf2)
    >>>>>000248 AF 2 1352: IP (tos 0x8, ttl 64, id 34356, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10f)
    >>>>>000223 AF 2 1352: IP (tos 0x8, ttl 64, id 34151, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x110)
    >>>>>005030 AF 2 128: IP (tos 0x8, ttl 61, id 45476, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf3)
    >>>>>000228 AF 2 1352: IP (tos 0x8, ttl 64, id 39765, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x111)
    >>>>>011247 AF 2 128: IP (tos 0x8, ttl 61, id 63692, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf4)
    >>>>>000226 AF 2 1352: IP (tos 0x8, ttl 64, id 29240, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x112)
    >>>>>000222 AF 2 1352: IP (tos 0x8, ttl 64, id 43306, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x113)
    >>>>>005663 AF 2 128: IP (tos 0x8, ttl 61, id 32980, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf5)
    >>>>>000228 AF 2 1352: IP (tos 0x8, ttl 64, id 56920, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x114)
    >>>>>010190 AF 2 128: IP (tos 0x8, ttl 61, id 3206, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf6)
    >>>>>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 4655, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x115)
    >>>>>000215 AF 2 1352: IP (tos 0x8, ttl 64, id 62740, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x116)
    >>>>>000203 AF 2 1352: IP (tos 0x8, ttl 64, id 35642, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x117)
    >>>>>006875 AF 2 128: IP (tos 0x8, ttl 61, id 37801, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf7)
    >>>>>000234 AF 2 1352: IP (tos 0x8, ttl 64, id 41803, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x118)
    >>>>>010651 AF 2 128: IP (tos 0x8, ttl 61, id 54256, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf8)
    >>>>>000235 AF 2 1352: IP (tos 0x8, ttl 64, id 30732, offset 0, flags
    >>>>>[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x119)
    >>>>>007913 AF 2 128: IP (tos 0x8, ttl 61, id 7647, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf9)
    >>>>>011166 AF 2 128: IP (tos 0x8, ttl 61, id 58037, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfa)
    >>>>>005483 AF 2 128: IP (tos 0x8, ttl 61, id 65275, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfb)
    >>>>>011250 AF 2 128: IP (tos 0x8, ttl 61, id 47289, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfc)
    >>>>>005505 AF 2 128: IP (tos 0x8, ttl 61, id 203, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfd)
    >>>>>104747 AF 2 128: IP (tos 0x8, ttl 61, id 45263, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfe)
    >>>>>8. 338674 AF 2 128: IP (tos 0x8, ttl 61, id 36351, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xff)
    >>>>>319992 AF 2 128: IP (tos 0x8, ttl 61, id 18085, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x100)
    >>>>>441837 AF 2 128: IP (tos 0x8, ttl 61, id 58323, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x101)
    >>>>>684077 AF 2 128: IP (tos 0x8, ttl 61, id 35487, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x102)
    >>>>>1. 167602 AF 2 128: IP (tos 0x8, ttl 61, id 34442, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x103)
    >>>>>2. 136032 AF 2 128: IP (tos 0x8, ttl 61, id 8345, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x104)
    >>>>>2. 984665 AF 2 128: IP (tos 0x8, ttl 61, id 35456, offset 0, flags
    >>>>>[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x105)
    >>>>>
    >>>>>
    >>>>>
    >>>>>
    >>>>>From what I'm seeing host B just stops sending without any reason. At
    >>>>
    >>>>least I don't see any fragmented packets. The only thing I've seen is
    >>>>some packets doesn't get ack'ed by the receiver.
    >>>>
    >>>>These packets never get ack'ed:
    >>>>46320:47548(1228)
    >>>>50004:51232(1228)
    >>>>53688:54916(1228)
    >>>>57372:58600(1228)
    >>>>61056:62284(1228)
    >>>>
    >>>>On host A I dumped the following:
    >>>>
    >>>>tcpdump -netttvvi gif6
    >>>>
    >>>>
    >>>>>1129985378.941282 AF 2 52: IP (tos 0x8, ttl 64, id 41637, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 574090240
    >>>>>490857876>
    >>>>>1129985378.952628 AF 2 1280: IP (tos 0x8, ttl 64, id 14004, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901
    >>>>>574090210>
    >>>>>1129985378.952657 AF 2 52: IP (tos 0x8, ttl 64, id 23243, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 574090251
    >>>>>490857901>
    >>>>>1129985378.958250 AF 2 1280: IP (tos 0x8, ttl 64, id 4306, offset 0,
    >>>>>flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901
    >>>>>574090210>
    >>>>>1129985378.971118 AF 2 1280: IP (tos 0x8, ttl 64, id 33534, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 490857920
    >>>>>574090229>
    >>>>>1129985378.971137 AF 2 52: IP (tos 0x8, ttl 64, id 60095, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 574090270
    >>>>>490857901>
    >>>>>1129985378.982488 AF 2 1280: IP (tos 0x8, ttl 64, id 11459, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931
    >>>>>574090240>
    >>>>>1129985378.982516 AF 2 52: IP (tos 0x8, ttl 64, id 33184, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 574090281
    >>>>>490857931>
    >>>>>1129985378.987989 AF 2 1280: IP (tos 0x8, ttl 64, id 54180, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931
    >>>>>574090240>
    >>>>>1129985378.994231 AF 2 1280: IP (tos 0x8, ttl 64, id 24535, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 490857942
    >>>>>574090251>
    >>>>>1129985378.994250 AF 2 52: IP (tos 0x8, ttl 64, id 30647, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 574090293
    >>>>>490857931>
    >>>>>1129985379.012101 AF 2 1280: IP (tos 0x8, ttl 64, id 61397, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 490857960
    >>>>>574090270>
    >>>>>1129985379.012132 AF 2 52: IP (tos 0x8, ttl 64, id 60550, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 574090311
    >>>>>490857960>
    >>>>>1129985379.017754 AF 2 1280: IP (tos 0x8, ttl 64, id 28408, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 490857961
    >>>>>574090270>
    >>>>>1129985379.023720 AF 2 1280: IP (tos 0x8, ttl 64, id 27558, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 490857972
    >>>>>574090281>
    >>>>>1129985379.023741 AF 2 52: IP (tos 0x8, ttl 64, id 21502, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 574090322
    >>>>>490857961>
    >>>>>1129985379.035333 AF 2 1280: IP (tos 0x8, ttl 64, id 18885, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
    >>>>>574090293>
    >>>>>1129985379.035362 AF 2 52: IP (tos 0x8, ttl 64, id 59875, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 574090334
    >>>>>490857984>
    >>>>>1129985379.040830 AF 2 1280: IP (tos 0x8, ttl 64, id 37252, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
    >>>>>574090293>
    >>>>>1129985379.046576 AF 2 1280: IP (tos 0x8, ttl 64, id 18349, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
    >>>>>574090293>
    >>>>>1129985379.046595 AF 2 52: IP (tos 0x8, ttl 64, id 43697, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 574090345
    >>>>>490857984>
    >>>>>1129985379.064961 AF 2 1280: IP (tos 0x8, ttl 64, id 38300, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013
    >>>>>574090322>
    >>>>>1129985379.064993 AF 2 52: IP (tos 0x8, ttl 64, id 47539, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 574090364
    >>>>>490858013>
    >>>>>1129985379.070688 AF 2 1280: IP (tos 0x8, ttl 64, id 30345, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013
    >>>>>574090322>
    >>>>>1129985379.076184 AF 2 1280: IP (tos 0x8, ttl 64, id 37536, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 490858014
    >>>>>574090322>
    >>>>>1129985379.076202 AF 2 52: IP (tos 0x8, ttl 64, id 34201, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 574090375
    >>>>>490858013>
    >>>>>1129985379.081680 AF 2 1280: IP (tos 0x8, ttl 64, id 20637, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 490858025
    >>>>>574090334>
    >>>>>1129985379.081709 AF 2 52: IP (tos 0x8, ttl 64, id 59866, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 574090380
    >>>>>490858025>
    >>>>>1129985379.087678 AF 2 1280: IP (tos 0x8, ttl 64, id 35213, offset
    >>>>>0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
    >>>>>64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 490858036
    >>>>>574090345>
    >>>>>1129985379.186906 AF 2 52: IP (tos 0x8, ttl 64, id 2465, offset 0,
    >>>>>flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
    >>>>>sum ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 574090486
    >>>>>490858036>
    >>>>
    >>>>tcpdump -netttvvi em1 host E.F.G.H
    >>>>
    >>>>
    >>>>>1129985379.064825 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
    >>>>>IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 45003, offset 0,
    >>>>>flags [none], length: 1352) E.F.G.H > A.B.C.D:
    >>>>>ESP(spi=0x0e0dffaa,seq=0x3e)
    >>>>>1129985379.065024 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
    >>>>>IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 1195, offset 0,
    >>>>>flags [none], length: 128) A.B.C.D > E.F.G.H:
    >>>>>ESP(spi=0x029a41b4,seq=0x2f)
    >>>>>1129985379.070572 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
    >>>>>IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 36820, offset 0,
    >>>>>flags [none], length: 1352) E.F.G.H > A.B.C.D:
    >>>>>ESP(spi=0x0e0dffaa,seq=0x3f)
    >>>>>1129985379.076069 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
    >>>>>IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 44971, offset 0,
    >>>>>flags [none], length: 1352) E.F.G.H > A.B.C.D:
    >>>>>ESP(spi=0x0e0dffaa,seq=0x40)
    >>>>>1129985379.076233 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
    >>>>>IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 56964, offset 0,
    >>>>>flags [none], length: 128) A.B.C.D > E.F.G.H:
    >>>>>ESP(spi=0x029a41b4,seq=0x30)
    >>>>>1129985379.081565 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
    >>>>>IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 24742, offset 0,
    >>>>>flags [none], length: 1352) E.F.G.H > A.B.C.D:
    >>>>>ESP(spi=0x0e0dffaa,seq=0x41)
    >>>>>1129985379.081741 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
    >>>>>IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 9390, offset 0,
    >>>>>flags [none], length: 128) A.B.C.D > E.F.G.H:
    >>>>>ESP(spi=0x029a41b4,seq=0x31)
    >>>>>1129985379.087562 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
    >>>>>IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 48065, offset 0,
    >>>>>flags [none], length: 1352) E.F.G.H > A.B.C.D:
    >>>>>ESP(spi=0x0e0dffaa,seq=0x42)
    >>>>>1129985379.186945 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
    >>>>>IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 36315, offset 0,
    >>>>>flags [none], length: 128) A.B.C.D > E.F.G.H:
    >>>>>ESP(spi=0x029a41b4,seq=0x32)
    >>>>
    >>>>If I'm not misleaded, this also doesn't show any errors except the
    >>>>missing ack's. host B just stops sending. If there's an ack missing,
    >>>>doesn't have the sending host to just repeat the un-ack'ed packet?
    >>>>
    >>>>The IPSec tunnel does not die. Even shortly after the (scp) transfer
    >>>>stalls the tunnel itself is still usable (for small amounts of data). To
    >>>>make it more worse, when disabling pf at the senders side, the transfer
    >>>>works. I've tripple checked pflog for denied packets on both sides but
    >>>>pf didn't filter any packets out.
    >>>>
    >>>>When disabling the IPSec rules using `setkey -F; setkey -FP' on the
    >>>>tunnel for a moment, the scp transfer does not stall. So it's not a gif
    >>>>issue.
    >>>>
    >>>>It doesn't seem to be an MTU issue (pf has also the rule 'scrub in/out
    >>>>all no-df'), but what kind of issue is that?? Has anybody ever
    >>>>experienced similar things? Or am I misinterpreting the tcpdump output?
    >>>>
    >>>>
    >>>>Any help and hint is appreciated! Without an error message I'm lost.
    >>>>
    >>>>Volker
    >>>>
    >>>>_______________________________________________
    >>>>freebsd-net@freebsd.org mailing list
    >>>>http://lists.freebsd.org/mailman/listinfo/freebsd-net
    >>>>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
    >>
    >>_______________________________________________
    >>freebsd-net@freebsd.org mailing list
    >>http://lists.freebsd.org/mailman/listinfo/freebsd-net
    >>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
    >
    >
    _______________________________________________
    freebsd-net@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-net
    To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"

  • Next message: Jayton Garnett: "Re: Problem with firewall"

    Relevant Pages

    • Re: Should I install Certificate Authority to solve these problems ?
      ... You can use IPsec with or without certs from your PKI. ... negotiations to your AD machines or those trusting the ... > In the item 1 below, the tool in use is a HP server management tool (type ... >>> Management is pushing to get Certificate Authority ...
      (microsoft.public.win2000.security)
    • Help! ipsec not talking IKE
      ... I'm trying to get ipsec working. ... host-to-host setup, with a PSK going as a test/proof of concept. ... machines are on the same subnet, but to avoid interfering with what's ...
      (comp.os.linux.security)
    • Re: IPSEC config
      ... spdadd 10.20.30.0/24 172.28.56.0/23 any -P out ipsec ... 15:24:18.927721 sunburn> acesfbsd: icmp: echo request ... fxp0: flags=8943mtu ... Then I have two machines on these nets that have routing pointing to ...
      (FreeBSD-Security)
    • Re: I have a few dumb questions
      ... on my DC and other machines in the domain, ... but if I disable IPSec on the DC and then on the other machines in the ... I forced a policy refresh, ... apparently missing a step even though I can visually verify that IPSec ...
      (microsoft.public.cert.exam.mcsa)
    • Re: Restrict Access to Domain Servers from Workgroup Computers
      ... client workstation not related to who is logged in. ... When you mention use of IPsec you identify the one current way ... disallowed to machines not allowed to access the shares (i.e. ...
      (microsoft.public.win2000.group_policy)