Re: jail issue

From: Jim Prewett (download_at_hpc.unm.edu)
Date: 02/14/04

  • Next message: Rostislav Krasny: "Re: 5.2.1-RC2 kernel panic 12"
    Date: Fri, 13 Feb 2004 21:34:26 -0700 (MST)
    To: Robert Watson <rwatson@freebsd.org>
    
    

    Hi Robert,

    I've been using jails (very happily) for quite some time and have *never*
    had a problem like this. I really don't have a clue what to look for :)

    I'm getting complaints from fellow keyserver ops as my IP seems to
    sometimes be the jail and sometimes the host, so some of my packets get
    rejected as that IP has not been configured (by the remote host) to be a
    peer. (how strange is that?!)

    Here is an email I recieved. I cvsup'd this morning, rebuilt everything,
    and did a final clean reboot before starting up the pgp jail. I recieved
    this email from one of my peer sites (the timestamps confirm this was
    after starting the jail after rebuilding):

    To: download@hpc.unm.edu
    Subject: PGP/nox again

    2004-02-13 10:52:01 Enabling gossip
    2004-02-13 10:52:02 Reconciliation attempt from unauthorized host
    <ADDR_INET
    129
    .24.244.72:2040>. Ignoring

    the host (nox) is 129.24.244.72, the jail (pgp) is 129.24.244.40.

    On Fri, 13 Feb 2004, Robert Watson wrote:

    >
    > On Fri, 13 Feb 2004, Jim Prewett wrote:
    >
    > > I run a PGP key server (SKS 1.0.6) inside of a jail. However, my key
    > > server seems to be getting confused as to its IP address and is sending
    > > packets as the host environment (not as the jail environment).
    >
    > Could you show the output of sockstat as run in the host environment?
    > Likewise, the output of ps ax. I'd like to see what the socket is bound
    > to, as the theory goes that jail modifies the bind requests from the
    > process to set them to the IP in the jail. Either we have a bug in socket
    > handling, or the process isn't running in the jail.

    I'm really afraid I may have inadvertantly found a bug! It is definantly
    in the jail environment (I've included the ps output below). The SKS
    daemons definantly answer on the jail environment IP (i've included the
    output of nmap against both the host and the jail below)!

    here are the sockets related to the sks process:

    nox# sockstat | grep sks
    root sks 276 5 tcp4 129.24.244.40:11371 *:*
    root sks 271 4 tcp4 129.24.244.40:11370 *:*
    root sks 276 6 stream ./db_com_sock
    root sks 271 5 stream ./recon_com_sock

    and sks processes:
    nox# ps ax | grep sks
     5804 p2 S+ 0:00.00 grep sks
      271 con- S+J 0:03.29 sks recon
      276 con- S+J 0:11.50 sks db

    nmap of host (nox) and jail (pgp):

    nox# nmap nox pgp -p 11370-11371

    Starting nmap 3.50 ( http://www.insecure.org/nmap/ ) at 2004-02-13 21:21
    MST
    Interesting ports on nox.hpc.unm.edu (129.24.244.72):
    PORT STATE SERVICE
    11370/tcp closed unknown
    11371/tcp closed pksd

    Interesting ports on pgp.hpc.unm.edu (129.24.244.40):
    PORT STATE SERVICE
    11370/tcp open unknown
    11371/tcp open pksd

    Nmap run completed -- 2 IP addresses (2 hosts up) scanned in 0.339 seconds

    ifconfig from the host:
    nox# ifconfig -a
    fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
            inet 10.0.0.1 netmask 0xff000000 broadcast 10.255.255.255
            inet6 fe80::2d0:b7ff:fe7f:f678%fxp0 prefixlen 64 scopeid 0x1
            ether 00:d0:b7:7f:f6:78
            media: Ethernet autoselect (none)
            status: no carrier
    vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
            inet 129.24.244.72 netmask 0xfffffc00 broadcast 129.24.247.255
            inet6 fe80::210:dcff:fedf:1a01%vr0 prefixlen 64 scopeid 0x2
            inet 129.24.244.40 netmask 0xffffffff broadcast 129.24.244.40
            ether 00:10:dc:df:1a:01
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lp0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> mtu 1500
    ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500
    sl0: flags=c010<POINTOPOINT,LINK2,MULTICAST> mtu 552
    faith0: flags=8002<BROADCAST,MULTICAST> mtu 1500
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
            inet 127.0.0.1 netmask 0xff000000

    ifconfig from the jail:
    pgp# ifconfig -a
    fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
            ether 00:d0:b7:7f:f6:78
            media: Ethernet autoselect (none)
            status: no carrier
    vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
            inet 129.24.244.40 netmask 0xffffffff broadcast 129.24.244.40
            ether 00:10:dc:df:1a:01
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lp0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> mtu 1500
    ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500
    sl0: flags=c010<POINTOPOINT,LINK2,MULTICAST> mtu 552
    faith0: flags=8002<BROADCAST,MULTICAST> mtu 1500
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

    If there is anything else that I can provide, please let me know. I'm
    *very* interested in resolving this.

    Thanks,
    Jim

    -- 
    James Prewett                           OpenPGP key: pub  1024D/31816D93
    Systems Team Leader		 	  Designated Security Officer
    HPC Systems Engineer III @ HPC@UNM -- download@hpc.unm.edu Jim@Prewett.org
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
    

  • Next message: Rostislav Krasny: "Re: 5.2.1-RC2 kernel panic 12"

    Relevant Pages

    • netstat/ipcs inside jail
      ... To resume quick, when i'm in a jail, netstat doesn't work properly. ... rl0: flags=8843mtu 1500 ... Opened by PID 68 ...
      (FreeBSD-Security)
    • Re: jail issue
      ... On Fri, 13 Feb 2004, Jim Prewett wrote: ... > sometimes be the jail and sometimes the host, so some of my packets get ... > and did a final clean reboot before starting up the pgp jail. ... > here are the sockets related to the sks process: ...
      (freebsd-stable)
    • Re: ipnat.conf - map and rdr wont work!
      ... provides a NAT within jail ... mtu 1500 ... this is not jail this is host environment ... this is on the hosting system hostname# and this is the client client# ...
      (freebsd-questions)
    • Jails - Pseudo Interfaces/ Virtual Networks - Best Practices?
      ... I'm in the process of planning a transition from VMWare (on CentOS host) hosting service-specific FreeBSD virtual machines to the seemingly more efficient, stable, and lower maintenance cost approach, now that I've figured out at least some of the "magic" of creating jails under FreeBSD. ... The questions generally relate to being able to consistently identify traffic from/to the jails, inside and outside the jail host, with tools such as ipfw and tcpdump. ... With VMWare, I was used to being able to create a pseudo-interface that could be "wired" to a virtual "switch" and then that switch could be bridged to a physical interface. ... Here DNS needs to be run for the host itself only, to be able to start up NTP and allow ssh connections for management over the "protected" interface. ...
      (freebsd-questions)
    • Re: carp+pfsync+freevrrpd+jail
      ... no traffic appears on master host - that means that the local carp ... host will answer the request. ... In case service IP is local it just dealing with jail. ... If only master carp answering packets then packet, coming from internal vlan200 comes to host: ...
      (freebsd-stable)