Found a case where this is actually "legal" - when you have
a piece of memory that is allocated without going to swap,
then lent, then swapped, then released before being read back
into memory. In this case, the release will clear the page without
notifying the swapper of such an event.
This case is mostly harmless -- the biggest problem is that we
leak swap due to pages that are released while swapped. This
tends to happen mostly with memory messages, and there is a high
re-use of addresses in this situation so I think the leakage
is not bad.
However, we continue to print the warning. If it turns out
that the leakage is cumulative, we'll have to come up with a
way to report this situation back to the swapper -- but it's
fairly disruptive, because it also forces us to do a full PT
walk and check on release plus send a message to the swap context
which is another chunk of code and syscall.
My working theory is this will be a "couple of pages" that circulate
around in a messaging pool inside swap memory, which is fine because
it doesn't impact main memory usage and swap memory is fairly
ample.
previously we could re-enter into a process through a divergent
call into the swapper. Now, we only ever re-enter as a syscall
through the syscall interface.
in order to make this happen we had to store all the thread and
stack state for the handler so we can do a re-entrant syscall, but
I think this is worth it. makes the code so much better.
- makes retrieve_page in-lineable into existing kernel routines
without a divergence issue on the call return
- reduces code duplication
- fixes a hairy bug on interrupt restore timing. With the previous
method, the interrupts would have to un-gate just before the
syscall executes its return to normalcy; this can cause a race
condition between a hardware interrupt and that return context
switch which can lead to the return context being corrupted.
By making this re-entrant, we can make the IRQ pause/restore
point happen outside of any fancy system context footwork, which
makes the interrupt via hardware source work again.
in practice the hard-OOM only seems to happen, and oom-doom
takes up precious memory that could be used for preventing
OOMs. Just leave one path in, but make it a bit smarter.
The smarts are still in userland, so the kernel is minimal
dependencies.
Re-entrant syscall is tricky because we end up smashing
the return thread context and stack of the current syscall.
We stash the thread context and stack in order to make this
happen, but we have to make some assumption about the potential
stack size. There's a assert to save us if this is wrong, but
generally the kernel is extremely well behaved on stack consumption
so we shouldn't be bumping into it.
This version of the OOM handler allows us to bury the OOM
handler "anywhere", which means we can handle the hard OOM
in multiple allocation methods without having to track the entry
and exit point of every single possible one.
This means we can chop a lot of fragile code out of the kernel.
We'll do that after this commit, and some testing, to make sure
this is the right path, but it feels "much nicer".
We'll probably also cut the oom-doomer, because I think in
practice it won't be effective and it consumes a non-trivial amount
of RAM, which would otherwise make OOMs more infrequent -- probably
an amount that would be greater than the benefit that any soft-OOM
handler can get us.
The Hard OOM re-entrant syscall can actually originate in
any process space, but only from inside the kernel. Modify the
checks so that the PID==2 check is applied only to all the other
calls; and that a supervisor mode check is applied to the hard OOM
call.
This version of the OOM handler uses a syscall in a syscall
(previously forbidden) to make the OOM case deep in an alloc
situation oomable, but also have multiple exit paths.
this saves us from having to recode oodles of exit paths which
is complicated and error-prone.
I also think we should trim back some of the OOM handling paths
because it is bloating the swapper, but let's commit the current
working state before we trim stuff.
transitioning this from a mere test stub into something more fully
fledged, but encountered a problem in the kernel along the way.
so, we're committing this pile of poop so we can swap over to
a new branch and put out the kernel fire before resuming with the
refactor.
UDMA Rx busy bit seems to get stuck occassionally. This seems
to be aggravated if we insert a dcache_flush in the poll loop.
The current time-out code seems to mitigate the problem, but
I have low confidencne that this is a 100% solution. I have a feeling
we'll be revisiting this again soon.
