I think this is safe to do because no strings can be directly
serialized into the PDDB's structures. The place to check for
this is inside the backend/ directory. For example, `DictName`
is where the name field is serialized into a structure that
goes onto the disk. It's explicitly length-checked.
What this means is that we could have calls into the pddb crate
with invalid length strings, however, if you check the lib
implementation, all of the APIs also pre-flight the length of
arguments to ensure they fit into the destination string.
So mostly what we're losing is the constraints on the intermediate
IPC representation, but the length is still checked on either side.
a bunch of lines are shorter or different length now which
triggers a mandatory reformat. this patch cleans that up,
so that going forward the manual rework patches are clearer and
not tied up with formatting patches.
there is one interesting thing that happens with the api
incompatibilities in that importing the KeyInit trait affects
both AES and SHA identically, so this trait now has to be imported
just at the function level instead of crate level because we don't
want it to affect both algorithms so broadly.
so...i think in this case, maybe the build.rs wasn't formatted?
it got picked up the second time when i ran the command.
in the case of pddbkey.rs, I think the changes made by rustfmt
were so dramatic it required two passes for it to become clean.
a lot of re-import warnings have cropped up. While I appreciate
their studiousness on this, the problem is that removing some
of these exports in the API crate could mean that future users
who want access to some API features that aren't used in this
configuration are now unaccessible.
I think it's not a huge problem at the moment, just spam-cleaning
the warnings because there are too many to deal with carefully
at the moment.
Mark `MemoryRange::as_slice()` and `MemoryRange::as_slice_mut()` as
unsafe.
This is due to the fact that not all representable types can be cast to
an arbitrary value. For example, if Process A sends random values to
Process B, then Process B casts the memory to a slice of `bool`, then
any values that are not 0 or 1 are undefined.
Because these functions can leak UB into the program, they are marked as
unsafe.
This patch adds `unsafe {}` blocks to all instances where this function
is called. Additionally, all instances have been checked to verify that
they do not cast to any invalid types -- most use `u8` or `u32`.
Additionally, the long-deprecated `MemoryRange::from_parts()` call was
removed, since that causes UB without being marked as unsafe.
Signed-off-by: Sean Cross <sean@xobs.io>
However, I didn't write this library (and I don't have a test
suite for it), so I'm not quite sure if this preserves the
invariants on the fd_mapping() structure. Needs @xobs' review.
I decided to go back on the idea of not changing the modals
and now all modals use the new larger font as well.
This creates issue #396, which I will deal with later once
I have the time to do some more testing. I'm pretty sure
some other dialog boxes are broken now too.
This change is made with the intention of making the xous-rs crate a bit
more freestanding without depending on the xtask build system.
Signed-off-by: Jean-Pierre De Jesus DIAZ <me@jeandudey.tech>
The resolution is to close the previously open basis if one with
the smae name is opened. This is in part because there isn't
a universally sensible/deterministic way of resolving behavior
in this case, and partially because the structure of the basis
map is a hash map keyed by name ... to differentiate basis with
same name/different password would require...leaking some information
about the password? i guess we could go strictly by opening order
by adding an opening parameter but basis management is already
confusing enough.
A message is popped up to indicate when the previous basis is
being unmounted.
1. track evictions as separate from cleanliness. not clean means
the data is dirty and needs writing back. eviction simply means
the data needs be re-retrieved from disk.
2. actually fill the data when we do a sync -- previously
we only copied the old data into the new page but then forgot
to allocate the data and assign it to the cache (oops)
3. sync the small pools after doing a remove inside the update()
command to make sure we've met invariants for future calls
to cache fills
so many corner cases, i'm going around in circles....
previously, we were invalidating at the first instance of
encountering the key; instead we should only invalidate once
we've processed it in our range of interest.
on valid pages were being updated in a dictionary entry as
it is updated; actually, entries that are invalid should also
be zeroized, otheriwse they "Stick around" as valid entries.
the new offset address was being used for copying data, instead
of the old offset, which was causing key data to refer to
an incorrect entry after deletion
Rather than hardcoding numbers, use an enum with the constant values.
This will make it easier to reason about what the constants mean.
Signed-off-by: Sean Cross <sean@xobs.io>
After a prune, key fill/scan can crash because there is an assumption
that data is always resident in the cache for a small key.
This code handles the case where the small key data was evicted during
a prune, and reads the data back in just for that key on demand.
Furthermore, during a `sync` call, it will pull in the key data
that corresponds to the page containing just the small key, allowing
that page to be re-composed and replaced in memory. caching is used
on the decryption of the small key page so the expensive bit
of the call should only happen once per small key bank.
tested with pddb-ci and manually in hosted mode.
A portion of code was re-written to make it clear that only the
length of allocated data is copied to/from disk and memory, instead
of the reserved space as the previous code would indicate. The reserved
space is guaranteed to be zero because the RAM-based page for composing
the small key block is zero-ized prior to operations; left comments
reminding me of this in the future so I don't accidentally optimize
something out and cause data leakage between small keys because I'm
allowing the reserved areas to contain stale data.
this fixes a problem where backups fail when autobacklight is
enabled, because autobacklight is constantly querying the PDDB
for the length of the autobacklight timeout, even after the
PDDB is unmounted.
this is to allow a UX process to clear the "waiting for boot"
indicator once an attempt has completed.
right now, it responds *after* the mount process is done,
so the "please waiting" text persists even doing a format.
I think that is...appropriate?
The initial wait message is now rendered by the first app
that owns the screen (currently shellchat).
The location where the pop-up modal is rendered is also now bracketed
directly around the mount call to avoid interference with other
edge cases that may pop up modals.
This commit shows two notifications:
1. as soon as the device boots, a notification greets the user and
tells them everything is fine, the device is booting
2. as soon as the user enters (or doesn't) enter the password, a
notification tell the user Xous is loading and mounting PDDB
I spent too much time on this but I believe it's a nice UX improvement.
if the buffer fills up to the point where there isn't enough
space for the 8-bytes of header, a split_at() call will fail before
the routine even gets to decide if the record can fit.
abort the loop if the buffer is that full.
results are slightly better on the first run (3700ms instead of 4900ms)
but there seems to be really excessive allocations using the heap
allocated serializer by rkyv. We really need to use the bufserializer
I think for this to work.
In general, hosted mode is assumed "by default" if the target
is not Xous. This allows packages to be built cleanly on command
lines that don't know to pass a --feature flag to specify a target.
However, an explicit "hosted" flag is still retained so that we
can confirm the build system intent. This saves us from the failure
mode that we thought we were building for target A, but accidentally
ommitted the flag, and instead we've been silently building for
"hosted" mode without any warning. The main consumer of the "hosted"
flag is now just the build.rs script inside of the utralib.
The implementation is a bit clumsy because the original
premise of the login loop is that it should be infalliable, and
because of the isolation of the password mechanism there isn't
a great way to pass number of failures back and forth across
that firewall (and I don't think there should be).
The application calling the password routine tracks failures
but the mounting routine has to kick that up to the main loop.
However, the PDDB main loop can't get stuck on the mount attempt
if you want to suspend, because it has to respond to the suspend
request.
So the failure message has to get kicked all the way back to the
status thread, which is what kicks off the mount, and then handled
there.
rkyv-derived enums throw warnings that they are unused.
Not sure how to get rid of them; I've tried surrounding them in
`#[allow(dead_code)]` directives, and also explicitly creating
each enum variant that I'm aware of through some dummy code and
some log prints of the dummy code calls to ensure the results are
used, and the warnings still persist.
Significantly, the warnings only happen for enums that take
a value. Simple rkyv enums do not throw a warning.
These warnings overwhelm the build logs, so they need to be isolated
in order for us to ensure that the build is clean of other more
important warnings.
The method chosen to isolate them is to take the enums that are
problematic, carve them into their own special file, and add
a file-wide `#![allow(dead_code)]` directive. This only happens
in 7 service crates, and this change commits all of the modifications
in a single big commit that could be undone later if this turns out
to be a compiler regression or something similar.
I also opened an issue with the rkyv maintainer to see what his
opinion is on the situation.
https://github.com/rkyv/rkyv/issues/318
every 1MiB block is checksummed with a sha512/256 truncated to
128 bits. This is used to confirm the media and transport integrity
only. It is not intended to be a cryptographic check, it just confirms
that the backup data was transferred correctly assuming no intentionally
malicious actors in the middle.
Most of the changes introduced in this commit were rolled back,
but, there is now a test facility that's hanging out for the PDDB
to help with future debug efforts.
The patch was motivated by a problem observed in FSCB consistency
when it's updated. Ultimately the root cause was traced to a
cache coherence problem and it's fixed by a patch in the spinor
crate.
However, this patch still retains a check that the data is coherent,
in case the flushing routine isn't complete or some other factor
was missed. It also includes a large amount of commented-out code
that keeps a bit of history used to debug the problem -- eventually,
we can just cull that but it's kept around in case we got the
root cause wrong (or there were multiple causes and coherence is
just one of them).
This will allow for e.g. xous on riscv32-imac but not a precursor
target, for example: by using feature selectors we get to have
a bigger range of targets that we support, such as for implementations
that haven't been upstreamed or have official target-triples.
The downsides are:
- patch override libraries don't absorb feature flags. so, for
example, the AES crate that is used to patch other crates won't
be aware of the target features without modifying the referencing
crates.
- features are normally meant to be additive, but target features
are mutually exclusive. The mutex is enforced by a check in
utralib that throws a panic on build if two targets are accidentally
selected at the same time.
This adds some UX feedback in the edge case that a suspend could not
happen due to a blocking sever, e.g. a sha512 hash pending or
a SPINOR flash write/erase pending.
The UI will tell the user to retry again; usually the ops in
a normally functioning system for hashes or SPI are very short,
so we could almost just retry automatically but I think for now
I want to see if this edge case actually happens.