So there's a bit of a nasty bug that arises from the GAM
both handling drawing primitives and context swaps.
In particular, because the GAM handles both of these in a
single thread, context swaps can't block, because the redraws
triggered by the context swap would be blocked, causing a deadlock.
When the API token for the predictor was introduced, this added
a blocking operation inside the context swap. This would cause
a deadlock under certain conditions.
This patch refactors the API token so that it's not a return
result of swapping the predictors. Instead, the predictors
return the API token as a non-blocking message to the GAM. This
API token is correlated with the *GAM* token (not the app token).
Present me thanks past me for adding the GAM token, which was
as of yet unused, but now, there is a very sensible role for it--
exactly what it was made for, e.g. handling GAM-local API calls
without disclosing the application token.
Anyways, hopefully this patch puts the race condition to
rest, while allowing us to have per-context prediction histories.
the inclusion of a custom rustflags in the .cargo/config
causes rust-analyzer in vscode to always trigger a rebuild of
xtask.
add a custom "check" directory for vscode. This grows the
size of the target directory :( but at least we're not
constantly rebuilding xtask.
The prediction API now takes an API token. This is a 128-bit
number that gets passed to the prediction engine, and it's meant
to be unique for every context that requires a predictor.
It's up to the prediction engine to do anything with the token.
For the "null" engines (such as the icontray for `vault` and
`tts`) the token is just ignored. But for example, in `shellchat`,
the API token is used to store a history record in a HashMap on
a per-token basis. This means that every app that uses the
`shellchat` history function gets their own history that can only
be read out by that app.
The actual token itself is mostly passed through the API. The
token storage is in the Context layer (each Context has a record
for a token, that starts as `None`), and is passed down to
the predictor every time the context swaps, and the predictor
receives the token and passes it back if the token was `Some`,
or allocates a new random token if it was `None`.
Note that everything is a Context, including, for example, menus and
modal dialog boxes, but not all contexts have predictors. Only
Contexts that ask for a predictor have them. This mechanism
works smoothly in this model (as opposed to other hacks and work-arounds
previously tried getting to this solution).
Mangement tokens were added to xous-names for the special case of
servers that had exactly 1 connection, so they could explicitly
control when they are disconnected (preventing e.g. griefing by
rogue processes).
This was a good idea except it also conflicts with the "eager TOFU"
model of Xous security when you have servers that act as plugins to
each other. Eager TOFU means Xous has a "trust on first use" model
for capabilities, and eager implies that anyone who will use
anything will claim it early on. This gives the important property
that one can ask, "has everyone eaten their TOFU?" before doing
anything really sensitive. If all the TOFU slots are used, then
you can conclude that no new rogue servers can inject themselves
into sensitive APIs and you can move forward.
In the case of a plugin API with multiple plug options, when
you unplug something, the previously eagerly occupied slot
now becomes unoccopied -- and this becomes a flag against
the TOFU check, disallowing sensitive operations to proceed.
This is *particularly* a problem with the predictive history
plugins for the IME. The architecture was created to allow
for multiple histories to be used. Up until now, only the shellchat
plugin was ever used, so, the eager TOFU check always passed.
However, in creating more applications, this flaw was uncovered
once we introduced a second plugin.
This means predictors can not delegate their security to Xous-names.
They must individually implement API tokens to control access
to the prediction history (to prevent rogue apps from e.g. connecting
to the predictor and then reading out the prediction history to
figure out what the most recent words were typed).
A subsequent commit will add the API tokens, but this commit simply
has xous-names return a mangament token even if servers with
more than 1 connection are created. The main downside of doing
this is the management token lose meaning in the context of
multiple servers; but, so long as the tokens aren't used for
anything in those situations, it's fine (cue burning building
meme).
More importantly, it allows the existing code to manage the xous-names
count to function even on a server that sets no explicit limit
on connection count, which is now going to be the case for the
prediction engines going forward.
still getting a heap allocation failure, something is
looking for 256k and even when the heap is increased
to a couple megabytes we get heap allocation failures.