Re: Removing kernel thread stack swapping

From: David Schultz (das_at_FreeBSD.ORG)
Date: 03/03/05

  • Next message: M. Warner Losh: "Re: Removing kernel thread stack swapping" 
    Date: Thu, 3 Mar 2005 10:35:05 -0500
To: John Baldwin <jhb@FreeBSD.ORG>

    On Thu, Mar 03, 2005, John Baldwin wrote:
    > On Thursday 03 March 2005 02:42 am, David Schultz wrote:
    > > Any objections to the idea of removing the feature of swapping out
    > > kernel stacks? Unlike disabling UAREA swapping, this has the
    > > small downside that it wastes 16K (give or take a power of 2) of
    > > wired memory per kernel thread that we would otherwise have
    > > swapped out. However, this disadvantage is probably negligible by
    > > today's standards, and there are several advantages:
    > >
    > > 1. David Xu found that some kernel code stores externally-accessible
    > > data structures on the stack, then goes to sleep and allows the
    > > stack to become swappable. This can result in a kernel panic.
    >
    > He found one instance.
    >
    > > 2. We don't know how many instances of the above problem there are.
    > > Selectively disabling swapping for the right threads at the
    > > right times would decrease maintainability.
    >
    > Probably 1. Note that since at least FreeBSD 1.0 programmers have had to
    > realize that the stack can be swapped out. The signal code in pre-5.x stores
    > part of the signal state in struct proc directly in order to support swapped
    > out stacks. In 5.x we just malloc the whole signal state directly since we
    > killed the u-area. sigwait() has a bug that should be fixed, let's not
    > engage in overkill and throw the baby out with the bath water. In general we
    > need to discourage use of stack variables anyway because when people use
    > stack space rather than malloc() space the failure case for running out is
    > much worse, i.e. kernel panic when you overflow your stack (even though KVM
    > may be available) vs. waiting until some memory is available or returning
    > NULL.
    >
    > Hence, don't kill this whole feature just because someone is too lazy to fix a
    > bug.

    Fair enough. I'll defer to you on the extent of the problem.
    David seemed to think that it was more widespread. (BTW, does
    *anyone* know what the PHOLD() in kern_physio is for? Is it a
    holdover from when the PCB was in struct user?)

    That still leaves my third point, which is that kernel stack
    swapping is no longer as effective as it was in 4.X. Resource
    hogs, particularly multithreaded ones, tend to get passed over by
    the swapper, so only the well-behaved processes (e.g. interactive
    ones) tend to get swapped out under high load. I have a WIP that
    I mentioned to you briefly before that fixes this by doing two
    things:

      a) The swapper sets a flag on threads that are unswappable.
         When I finish this, those threads will swap themselves out
         the next time they try to enter or exit the kernel.

      b) Individual threads within a process can be swapped out; they
         don't all have to be swapped out at the same time.

    I'm not actually sure if (b) is a good thing to do. For many
    applications, swapping out one thread will just cause all the
    others to quickly stall while waiting for it. Thoughts?

    In any case, I have no time to finish it right now, but assuming I
    don't get to axe it all, I'll get to finishing it eventually...
    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: M. Warner Losh: "Re: Removing kernel thread stack swapping"

    Relevant Pages

    • Removing kernel thread stack swapping
      ... Any objections to the idea of removing the feature of swapping out ... wired memory per kernel thread that we would otherwise have ... stack to become swappable. ...
      (freebsd-arch)
    • Re: Removing kernel thread stack swapping
      ... Unlike disabling UAREA swapping, this has the ... >> wired memory per kernel thread that we would otherwise have ... >> stack to become swappable. ...
      (freebsd-arch)
    • Re: Removing kernel thread stack swapping
      ... > Any objections to the idea of removing the feature of swapping out ... Unlike disabling UAREA swapping, this has the ... > wired memory per kernel thread that we would otherwise have ... > stack to become swappable. ...
      (freebsd-arch)
    • [Full-disclosure] PHRACK 64: ATTACKING THE CORE
      ... - The Slab Allocator ... - Slab overflow exploiting: ... - Forcing a kernel path to sleep ... - Stack Frame Flow Recovery ...
      (Full-Disclosure)
    • Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
      ... stack usages for you is that they contain a 'cpumask_t' on the stack. ... We can enable MAXSMP and raise the CPU limits some time in the future. ... not accept a specially built kernel, but only a kernel that has been ... know how extensively these distributions test and certify for many known ...
      (Linux-Kernel)