IP_BOUND_IF

Solaris 10 update 6, no additional patches.

I'm trying to make Apache send outbound TCP packets through the same
interface on which incoming packages were received. It seems IP_BOUND_IF
socket option can do the trick, although ip(7p) man page is somewhat
unclear:

IP_BOUND_IF Limit reception and transmission of pack-
ets to this interface. Takes an integer as
an argument. The integer is the selected
intefrace index.

There is also PSARC 2007/565 rationalization:

http://mail.opensolaris.org/pipermail/opensolaris-arc/2007-October/004337.html

But it sort of assumes you already know what IP_BOUND_IF does. I suppose
it bypasses the routing table and sends the packets directly to the
specified interface.

So I have a server with two interfaces:

bge0: 192.168.100.130
bge1: 192.168.50.201

I'm seeing packets coming in through bge1, but leaving through bge0,
regardless of IP_BOUND_IF setting. The default route for the client points
to bge0, so that should be normal without IP_BOUND_IF. I was hoping
IP_BOUND_IF would make packets leave thorugh bge1, as well.

As far as my debug logs show, I'm passing the correct arguments to
setsockopt() and it returns zero. This is being done after bind() and
listen() have been called. There is a possibility that IP_BOUND_IF should
have been set before one or both of these calls to get the effect I'm
looking for, but the man page doesn't say anything about that.

The option is being set only on the listening sockets and perhaps sockets
returned with accept() don't inherit the setting, but setting on listen
socket should be good enough for the first SYN-ACK packet.

pfiles don't show any difference for those sockets after IP_BOUND_IF is
set. I don't know if there should be anything shown. Here is a sample for
file descriptor 7:

7: S_IFSOCK mode:0666 dev:346,0 ino:6922 uid:0 gid:0 size:0
O_RDWR|O_NONBLOCK
SOCK_STREAM
SO_REUSEADDR,SO_KEEPALIVE,SO_SNDBUF(49152),SO_RCVBUF(49152),IP_NEXTHOP(0.0.192.0)
sockname: AF_INET 192.168.50.201 port: 8000

I tried calling getsockopt() after sucessfull setsockopt(), just for
debugging purposes and it also returns zero, but the parameter it returns
is always zero. That shouldn't be so. Here is an exempt from the debug log:

[Thu Nov 05 20:18:26 2009] [info] searching IF for fd 7, IP c0a832c9
[Thu Nov 05 20:18:26 2009] [info] setsockopt(7, IP_BOUND_IF, 3)
[Thu Nov 05 20:18:26 2009] [info] optlen = 4
[Thu Nov 05 20:18:26 2009] [info] getsockopt(7, IP_BOUND_IF) = 0

The "= 0" in getsockopt line is returned optval, not function return
value. Interface index 3 is bge1 (192.168.50.201).

I don't know what to make of that.

I tried messing around with mdb in hope I'd be able to check kernel
structures to see if setsockopt() call changed anything, but I don't know
how to do that.

00000600229eed00::pfiles
FD TYPE VNODE INFO
0 CHR 00000600117f7800 /devices/pseudo/mm@0:null
1 CHR 00000600117f7800 /devices/pseudo/mm@0:null
2 REG 000006001885e100 /opt/csw/apache2/var/log/error-prefork_log
3 DOOR 000006001291eb00 /var/run/name_service_door [door to 'nscd' (proc= 60014a62c60)]
4 SOCK 0000060020ab3740 socket: AF_INET 192.168.100.130 80
5 SOCK 00000600183978c0 socket: AF_INET 192.168.100.130 8000
6 SOCK 0000060019975100 socket: AF_INET 192.168.50.201 80
7 SOCK 0000060021c04100 socket: AF_INET 192.168.50.201 8000
8 FIFO 0000060014bff4c0
9 FIFO 0000060014bff3c0
10 REG 0000060018068a80 /opt/csw/apache2/var/log/morrigan-8000.error_log
11 REG 000006001885e100 /opt/csw/apache2/var/log/error-prefork_log
12 REG 0000060011e73600 /opt/csw/apache2/var/log/morrigan-80.error_log
13 REG 0000060018895d40 /opt/csw/apache2/var/log/access-prefork_log
14 REG 000006001fd76980 /opt/csw/apache2/var/log/morrigan-8000.access_log
15 REG 000006001fd76880 /opt/csw/apache2/var/log/morrigan-80.access_log
16 REG 0000060023abc340 /tmp/aprzkayfy
17 REG 0000060020c2c940 /tmp/session_mm_apache2handler0.sem
18 REG 0000060020e90680 /opt/csw/apache2/var/run/accept-prefork.lock.12331

I don't know which command to use to display socket structure(s) for file
descriptor 7.

And now I'm stuck. Can anybody help?

--
.-. .-. Yes, I am an agent of Satan, but my duties are largely
(_ \ / _) ceremonial.
|
| dave@xxxxxxxxxxxxxx
.

Relevant Pages

  • Re: [patch 4/10] s390: network driver.
    ... but it seems someone is complaining about some behavior changing? ... network driver discard packets on link-down. ... However this approach doesnt play well if the socket can ... be blocked completely because of /one/ interface having its link ...
    (Linux-Kernel)
  • Re: SO_BINDTODEVICE
    ...     in the passed interface name. ...     processed by the socket. ... does not mean that if you bind to an Ethernet interface, only packets ...
    (comp.os.linux.networking)
  • Re: python libpcap equivalent
    ... socket to send them out again on whatever interface you want. ... under Linux you can also capture packets using a raw-mode ... user-space app to "pretend" to be an Ethernet interface in the ...
    (comp.lang.python)
  • Socket Programming: How to terminate a thread "listening" for UDP packets?
    ... I have started out with socket programming by developing a little tool that will read a file transfer from telemetry. ... The first version is very rudimentary, meaning I do not control correct sequence of packets and missing packets, I just dump everything to a file. ... What I am trying to do is - I want to allow the user in a basic GUI to click on a button and toggle the program status between "listening" and "not listening" for new file transfers / packets. ...
    (comp.unix.programmer)
  • raw sockets and blocking
    ... cable of one interface has been pulled? ... which uses a single raw, AF_INET/OSPF socket and manages it's own IP ... to send/receive OSPF packets to/from a number of interfaces. ...
    (Linux-Kernel)