Commit graph

392 commits

Author SHA1 Message Date
bunnie
5370c333da add callback for monitoring basis lock/unlock events 2022-12-29 00:10:42 +08:00
bunnie
ad79ebf8ba chore: more typos. no really, i can stop at any time. 2022-12-13 23:02:09 +08:00
bunnie
b1b6d46d27 chore: more typos
probably should not have tried to spell check this, now i'm wasting
my time.
2022-12-13 22:56:37 +08:00
bunnie
52955681d5 reflect unmount status after tryunmount
this fixes a problem where backups fail when autobacklight is
enabled, because autobacklight is constantly querying the PDDB
for the length of the autobacklight timeout, even after the
PDDB is unmounted.
2022-12-02 20:10:10 +08:00
bunnie
c3ba30d55d expand on errors reported 2022-11-09 14:25:26 +08:00
bunnie
0646ec23ba track "attempted" mounts in PDDB
this is to allow a UX process to clear the "waiting for boot"
indicator once an attempt has completed.

right now, it responds *after* the mount process is done,
so the "please waiting" text persists even doing a format.

I think that is...appropriate?
2022-11-07 00:21:52 +08:00
bunnie
3ee36f41ee remove UX delays on PDDB path
retain just one of them going into the erase routine, because
it consumes enough CPU that the dialog box takes a bit too long to
pop up.
2022-11-06 18:45:17 +08:00
bunnie
fa90e74a10
Merge pull request #270 from gsora/feat/pddb-init-notification
feat: show boot process notifications for PDDB
2022-11-06 03:40:17 +08:00
bunnie
7eecf2b1ce remove the initial wait from PDDB; move the pop-up modal location
The initial wait message is now rendered by the first app
that owns the screen (currently shellchat).

The location where the pop-up modal is rendered is also now bracketed
directly around the mount call to avoid interference with other
edge cases that may pop up modals.
2022-11-06 03:07:35 +08:00
Gianguido Sorà
d277ae9551 feat: show boot process notifications for PDDB
This commit shows two notifications:
 1. as soon as the device boots, a notification greets the user and
 tells them everything is fine, the device is booting
 2. as soon as the user enters (or doesn't) enter the password, a
 notification tell the user Xous is loading and mounting PDDB

I spent too much time on this but I believe it's a nice UX improvement.
2022-11-04 17:31:27 +01:00
bunnie
5de44a9502 don't fail if keyattribute not found in current basis when no basis specified
This is not actually an error, we just need to try another basis
down the list. This is a copypasta bug that was only caught now. :-/
2022-11-04 17:05:51 +08:00
bunnie
0d9a39a349 print debug information when a key goes missing in bulk read
this helps debug problems down the road
2022-11-04 17:05:32 +08:00
bunnie
d74673ab29 allow main menu to be used if there are no root keys 2022-11-04 03:55:57 +08:00
bunnie
80a1bd3654 resolve issue #267 2022-11-03 23:18:13 +08:00
bunnie
ff2a7f8240 some more debugging to help trace through dict sync issues 2022-11-03 21:06:04 +08:00
bunnie
b04dcc21ef add smalldb flag, which builds a PDDB with a size of 4MiB
do not interchange this on devices with the full sized
PDDB, you may corrupt the database
2022-11-03 21:04:14 +08:00
bunnie
719020b08b don't allow polls to the unmount to turn true until other callbacks have been allowed 2022-11-03 17:23:11 +08:00
bunnie
7af04b7000 a bit more debugging around dictionary creation 2022-11-03 01:38:18 +08:00
bunnie
d63c452fb6 catch serialization buffer edge case
if the buffer fills up to the point where there isn't enough
space for the 8-bytes of header, a split_at() call will fail before
the routine even gets to decide if the record can fit.

abort the loop if the buffer is that full.
2022-10-27 23:00:10 +08:00
bunnie
c82a79dab9 add performance monitoring in for bulk reads 2022-10-27 16:45:04 +08:00
bunnie
bef734553e clarify some comments and clean up debug messages 2022-10-27 02:53:38 +08:00
bunnie
6992b31104 fix the bufserializer approach
this is much more efficient than the alloc serializer
2022-10-27 02:41:46 +08:00
bunnie
6272a94604 intermediate attempt to extract a length parameter 2022-10-27 01:57:42 +08:00
bunnie
ac01088ec7 Merge branch 'pddb-bulk' into pddb-bulk-bufserializer 2022-10-27 01:30:27 +08:00
bunnie
3d1885e165 add test code
results are slightly better on the first run (3700ms instead of 4900ms)
but there seems to be really excessive allocations using the heap
allocated serializer by rkyv. We really need to use the bufserializer
I think for this to work.
2022-10-27 01:10:17 +08:00
bunnie
721dbc82d7 add an attempt to use bufserializer that didn't work 2022-10-26 23:29:43 +08:00
bunnie
d9bffb3556 add some notes about making this more efficient 2022-10-26 22:06:00 +08:00
bunnie
665e7dce51 fix minor bugs, now passing basic test 2022-10-26 15:16:40 +08:00
bunnie
bffe92cab9 add first draft of bulk serialize/deserialize functions 2022-10-26 05:29:44 +08:00
bunnie
2e768dd19e Merge branch 'main' of github.com:betrusted-io/xous-core into pddb-bulk 2022-10-25 23:15:51 +08:00
bunnie
9e39c432bd Merge branch 'main' into pddb-bulk 2022-10-25 17:41:58 +08:00
bunnie
f5289be6fe experiment with 'std' in rkyv for pddb crate 2022-10-25 17:41:12 +08:00
bunnie
0ec75861e6 cleanup a normalization that escaped due to merge conflict 2022-10-25 17:40:18 +08:00
bunnie
30ea893c74
Merge branch 'main' into xous-rs-os-selection 2022-10-25 17:28:33 +08:00
bunnie
67fc21fbdd harmonize hosted mode selection, but retain an explicit hosted flag
In general, hosted mode is assumed "by default" if the target
is not Xous. This allows packages to be built cleanly on command
lines that don't know to pass a --feature flag to specify a target.

However, an explicit "hosted" flag is still retained so that we
can confirm the build system intent. This saves us from the failure
mode that we thought we were building for target A, but accidentally
ommitted the flag, and instead we've been silently building for
"hosted" mode without any warning. The main consumer of the "hosted"
flag is now just the build.rs script inside of the utralib.
2022-10-25 17:21:02 +08:00
bunnie
97f0d34daa more analytics on PDDB debugging 2022-10-24 02:35:55 +08:00
bunnie
83d31ff8b9 fix non-perf builds for pddb 2022-10-24 00:37:05 +08:00
bunnie
e515092636 add performance profiling options to PDDB 2022-10-23 23:14:57 +08:00
bunnie
fc8f2a25e8 expand on "internal error" for .get() results from PDDB 2022-10-20 17:40:11 +08:00
bunnie
d7392d3704 improve comments and add count to the user-failed mount case 2022-10-17 03:51:36 +08:00
bunnie
de41e5ecff rework password login to have a count and reboot/suspend after N retries
The implementation is a bit clumsy because the original
premise of the login loop is that it should be infalliable, and
because of the isolation of the password mechanism there isn't
a great way to pass number of failures back and forth across
that firewall (and I don't think there should be).

The application calling the password routine tracks failures
but the mounting routine has to kick that up to the main loop.
However, the PDDB main loop can't get stuck on the mount attempt
if you want to suspend, because it has to respond to the suspend
request.

So the failure message has to get kicked all the way back to the
status thread, which is what kicks off the mount, and then handled
there.
2022-10-17 02:40:51 +08:00
bunnie
ec1d1b4444 don't try to unmount anything if nothing was mounted 2022-10-17 00:32:06 +08:00
bunnie
f815ed85b5 fix rust 1.64.0 rkyv warning regressions on unused enum types
rkyv-derived enums throw warnings that they are unused.
Not sure how to get rid of them; I've tried surrounding them in
`#[allow(dead_code)]` directives, and also explicitly creating
each enum variant that I'm aware of through some dummy code and
some log prints of the dummy code calls to ensure the results are
used, and the warnings still persist.

Significantly, the warnings only happen for enums that take
a value. Simple rkyv enums do not throw a warning.

These warnings overwhelm the build logs, so they need to be isolated
in order for us to ensure that the build is clean of other more
important warnings.

The method chosen to isolate them is to take the enums that are
problematic, carve them into their own special file, and add
a file-wide `#![allow(dead_code)]` directive. This only happens
in 7 service crates, and this change commits all of the modifications
in a single big commit that could be undone later if this turns out
to be a compiler regression or something similar.

I also opened an issue with the rkyv maintainer to see what his
opinion is on the situation.

https://github.com/rkyv/rkyv/issues/318
2022-10-09 17:17:20 +08:00
bunnie
2ff9163789 add checksums to PDDB backup
every 1MiB block is checksummed with a sha512/256 truncated to
128 bits. This is used to confirm the media and transport integrity
only. It is not intended to be a cryptographic check, it just confirms
that the backup data was transferred correctly assuming no intentionally
malicious actors in the middle.
2022-10-09 01:10:42 +08:00
bunnie
62af36560b checks for cache coherence issues
Most of the changes introduced in this commit were rolled back,
but, there is now a test facility that's hanging out for the PDDB
to help with future debug efforts.

The patch was motivated by a problem observed in FSCB consistency
when it's updated. Ultimately the root cause was traced to a
cache coherence problem and it's fixed by a patch in the spinor
crate.

However, this patch still retains a check that the data is coherent,
in case the flushing routine isn't complete or some other factor
was missed. It also includes a large amount of commented-out code
that keeps a bit of history used to debug the problem -- eventually,
we can just cull that but it's kept around in case we got the
root cause wrong (or there were multiple causes and coherence is
just one of them).
2022-10-07 18:52:57 +08:00
bunnie
4e40ae6c5d move from target selectors to feature selectors for back-ends
This will allow for e.g. xous on riscv32-imac but not a precursor
target, for example: by using feature selectors we get to have
a bigger range of targets that we support, such as for implementations
that haven't been upstreamed or have official target-triples.

The downsides are:

- patch override libraries don't absorb feature flags. so, for
example, the AES crate that is used to patch other crates won't
be aware of the target features without modifying the referencing
crates.

- features are normally meant to be additive, but target features
are mutually exclusive. The mutex is enforced by a check in
utralib that throws a panic on build if two targets are accidentally
selected at the same time.
2022-10-02 02:56:44 +08:00
bunnie
85279566f0 fix #77
This adds some UX feedback in the edge case that a suspend could not
happen due to a blocking sever, e.g. a sha512 hash pending or
a SPINOR flash write/erase pending.

The UI will tell the user to retry again; usually the ops in
a normally functioning system for hashes or SPI are very short,
so we could almost just retry automatically but I think for now
I want to see if this edge case actually happens.
2022-09-24 02:40:11 +08:00
bunnie
9225cbaba3 resolve issue #206 by adding check for existing basis 2022-09-20 23:06:22 +08:00
bunnie
4ae3308fbe add PIN change to the PDDB 2022-08-12 23:10:18 +08:00
bunnie
2b4ce52a10 fix some regressions on empty key listings 2022-08-02 02:52:04 +08:00
bunnie
3d992cedf4 fix an import error + reduce log noise on a normal path 2022-08-02 01:38:07 +08:00
bunnie
0d61cbb838 improve debug output 2022-08-02 01:37:45 +08:00
bunnie
13b366345c clean up/optimize the key listing routine
provides some marginal improvement on the vault UX
2022-08-01 23:12:26 +08:00
bunnie
826aa43f20 Merge branch 'pddb-flamegraph' into main 2022-08-01 15:46:57 +08:00
bunnie
5be62e372d chore: move AES-GCM-SIV off of branch to 0.11.1 official
this patch moves us off the branch we had to hop on for cipher
API compatibility to the one that's officially published by
RustCrypto. This closes #168.
2022-08-01 15:44:16 +08:00
bunnie
7d62ca85b8 some preliminary schwags at making flamegraphs
turns out, they're pretty useless. I mean, for debugging actual
running code. it tells us basically what we know already which is
99.99% of the time is spent waiting on inter-thread messaging
in hosted mode.
2022-08-01 01:32:19 +08:00
bunnie
e469736278 stagger the flush timing a little bit
noticed it's hitting exactly when NTP updates because it also
happens on a 24-hour schedule, staggering a bit to try and keep
everything from hitting exactly at the same time once a day.
2022-07-29 15:00:44 +08:00
bunnie
46c53d9612 add telemetry for better PDDB CI 2022-07-28 18:46:56 +08:00
bunnie
ac1f746566 add fscb/spaceupdate journal flushing & periodic flush
- fix journal number bug in basis root record
- fix conflict PTE resolution to also consider basis root record,
which has a different encryption form.
2022-07-28 04:54:35 +08:00
bunnie
ec5cc29ccd hotfix for incorrect AES-KWP implementation
The crate we originally grabbed for AES-KWP seems to be non-compliant
(or I'm an idiot and I can't figure out how to call it in a way
that is compliant).

See https://github.com/jedisct1/rust-aes-keywrap/issues/9

Turns out that `aes_kw` *is* compliant, but unfortunately, it didn't
exist back when the PDDB was being written. The PDDB started in October
2021, and `aes_kw` was pushed in Jan 2022. *just* barely missed it.

Anyways, caught this error while trying to implement some CI for
backups. Hurray for testing!

This patch removes the encryption path using the incorrect version
of the algorithm, but leaves the decryption path in place. The
system will try to decrypt initially with the correct version,
but if that fails, it will fall back and try the incorrect version.
If that succeeds, it returns an error code with the correct key and
an updated wrapped key, which the caller can store so that future
calls do not fail. Thus, we have a seamless migration from the
incorrect version to the correct version.

Have verified that the aes_kw version *does* work with the CI
script, insofar as it has been written (we just got to that point
and got stuck). So, full speed ahead. Users shouldn't notice this
change.

Also, fwiw, I did take a look at the "incorrect" implementation,
and it doesn't look obviously insecure. Printing out the various
rounds of the wrapping function looks like the first couple rounds
match, but it diverges somewhere probably due to an off-by-one
in a counter or something like that. I didn't chase it out -- just
wanted to make sure that nothing looked like a huge data compromise.

Good news is this comes right on the heels of v0.9.9 release, so
this will be pushed as a hotfix right away.
2022-07-25 02:56:20 +08:00
bunnie
6f10061653 reset "don't ask again" flag for password creation during factory test
this facility is necessary in case a worker accidentally picks
the wrong setting, disabling the feature for the device. When
the factory test completes, a call is made to reset the parameter.

This was a slightly complicated affair because the the don't ask
flag is contained within the rootkeys enclave itself, and we can't
just give access to rootkeys willy-nilly. So the flag is passed
through the PDDB and then to the rootkeys enclave. It's not harmful
to have it reset by an unauthorized actor; the pop-up will not
happen if passwords are, in fact, already set.
2022-07-19 23:57:25 +08:00
bunnie
e40a109f6c add UX flow dedicated to the 'churn' operation 2022-07-19 16:57:47 +08:00
bunnie
9fc74e3291 strip out debug info 2022-07-19 15:56:16 +08:00
bunnie
e8188309cf add rekey operation for PDDB migrations to new devices 2022-07-19 03:16:37 +08:00
bunnie
a89ae13463 move pddb mounter to status thread 2022-07-11 04:12:44 +08:00
Sean Cross
d6ac900748 pddb: backend: use Instant for access time
Instead of using the ticktimer server directly, use
`std::time::Instant::now()`. This enables tests to be run in hosted
mode.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-07-06 13:59:35 +08:00
Sean Cross
086c014de2 pddb: close all related filehandles when a file is deleted
Signed-off-by: Sean Cross <sean@xobs.io>
2022-07-05 23:56:31 +08:00
Sean Cross
ee18fe889c pddb: initial libstd support
Add support for libstd calls to pddb.

The pddb uses `:` as the path separator, and grafts the concept of paths
onto dicts.

Currently implemented features:

    * mkdir
    * rmdir
    * create file
    * open file
    * read file
    * write file
    * unlink file
    * seek file

Other calls may be added as needed.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-07-05 22:50:58 +08:00
Sean Cross
6ae91d8e46 pddb: make select_basis and latest_basis non-mut
These do not need to clone data and do not need to be mut. This improves
access time considerably.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-07-05 17:23:40 +08:00
bunnie
e26106e334 fix warnings and clean up configs
tighten up the scope of some features to reduce warning messages
2022-07-04 18:13:11 +08:00
bunnie
8565c7efe1 fix FSCB generation issues
- fix problem where we were generating a size based off of total disk
size, instead of fraction of available space

- enumerate FSCB entries into the page table scan. This is because
the page table cache could hold used entries that aren't reflected
in the low-level disk scan, and there is no way to trigger a
flush at the point of re-allocation.
2022-07-04 17:56:54 +08:00
bunnie
baa5097313 fix basis FSCB generation on full - basic fix in place and tested 2022-07-04 15:19:40 +08:00
bunnie
7162b05a7f add routines for testing secret basis
When the "autobasis" feature is selected, the hardware block
will track all basis that have been opened and unlock them when
it comes time to scan/generate more FSCB space. It also bypasses
all UX normally invoked.

This allows us to create tests that open and close Bases automatically,
but of course, totally destroys the notion of plausible deniability.
So, don't turn the feature on for production.
2022-07-04 04:06:53 +08:00
bunnie
8a81954cf0 clean up the page table enumeration
since all the keys are now disclosed, the routine has been
renamed accordingly and the re-scan of existing bases is
redundant.
2022-07-03 01:12:12 +08:00
bunnie
559229d85e fix key enumeration to include keys that we already know 2022-07-03 01:06:56 +08:00
bunnie
e8140f257c make sure the basis list is displayed at least once
need to cover the case that no enumeration was done, and just
show a "null" list confirming basically nothing other than
the system basis is disclosed.
2022-07-03 00:42:16 +08:00
bunnie
b10cf878db fix large key read; and cursor positions in read() and write()
1. the large key read algorithm was broken. this is now fixed. we only
saw this triggered when the large key was much larger than 8kiB (and
the original CI tests just use 8kiB keys for "large").

2. the cursor position in read() and write() were being incremented
by the expected read amount, not the actual read amount.
2022-07-03 00:35:11 +08:00
bunnie
587f7ef39c some enhanced debugging 2022-07-02 00:09:38 +08:00
bunnie
ae37c2cd02 don't truncate key allocations that have an alloc_hint specified 2022-07-02 00:09:21 +08:00
bunnie
a1950a4c00 make a default alloc hint that's not 0
set to 8 bytes initially, could be adjusted up or down.
2022-07-01 19:45:55 +08:00
bunnie
5008c7afac first draft of all basis unlock
This has worked in testing, but:

- There seems to be a new error unrelated to this with allocating
very large blocks of memory, which needs to be traced down.
- Because this operation will typically fully block the caller,
UX redraws can't happen (unless the caller has a fully multithreaded
redraw routine). This needs some thought because:

1. If you have several basis you're trying to unlock, eventually,
you'll overflow the application incoming queue with redraw requests,
and the system will crash.

2. It's cosmetically disturbing to have the dialog boxes piling up
on top of each other during a critical operation like this.

I *think* the right thing to do is to refactor the GAM a bit
to handle pop-up modals that block a bit more gracefully. But
I think it's more urgent to trace down the problem with large blocks
of memory being allocated, since it is a functional issue and not
a cosmetic one.
2022-07-01 17:09:51 +08:00
bunnie
23a31c2cf0 fix PDDB initial stack size and increase heap limit on boot
I had thought there was an issue with heap size limit increases
and multithreading but I think actually the problem is the
PDDB runs pretty close to the stack size limit. This increases
the stack size limit for the PDDB on boot and also bumps up
the heap limit so caching can work more flawlessly.
2022-06-28 19:58:20 +08:00
Sean Cross
7aae6dcab9 pddb: change Option<Opcode> -> Opcode in main
Instead of writing `Some(Opcode::*)` repeatedly, change it to simply
match against `Opcode::*`. This is accomplished by defining a new opcode
`Invalid` that will be matched against as a default.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-06-27 09:24:11 +08:00
Sean Cross
748d4ce87d Merge remote-tracking branch 'origin/main' into libstd-fs 2022-06-27 09:14:04 +08:00
Sean Cross
8cd04c3415 pddb: rework PddbBuf ordering to be non-packed
The new ordering remains the same size, but is no longer
`#[repr(packed)]`. This allows it to be `#[repr(align(4096))]` which
means we can get away with using a stack page for this struct instead of
allocating a new object every time.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-06-27 09:13:00 +08:00
bunnie
fd63a33078 add PDDB heap limit increase based on passing a threshold 2022-06-27 04:03:33 +08:00
Sean Cross
1388da7680 Merge remote-tracking branch 'origin/vault' into libstd-fs 2022-06-26 16:48:20 +08:00
bunnie
88ea0145da clean up some debug spew 2022-06-26 03:55:26 +08:00
bunnie
9f409fda2d fix basis search order issues
at least, it passes some simple tests. much more testing is needed.
2022-06-26 03:52:27 +08:00
bunnie
e62c9bf3d3 add basis management functions and debug some basis issues
there's some problems with the automatic basis unionization
feature. now debugging...
2022-06-26 02:11:13 +08:00
Sean Cross
19532c5634 pddb: add explicit discriminants to enums
In order to ensure the API is stable, add explicit opcodes to enums
within pddb.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-06-25 11:00:41 +08:00
bunnie
4c18662430 add a periodic scrub to the PDDB
this periodically polls the PDDB and has it check the memory allocation
levels, and if too high, invokes a scrub attempt.

This bit is probably the most poorly tested of all the code, but
also some of the simpler logic. The more important part this code
brings is analytics on actual heap and cache usage, which will help
inform us on how urgent/bad things are getting in terms of PDDB
cache usage.
2022-06-17 02:02:11 +08:00
bunnie
dc0a4e5400 add backpressure triggers to the CI tests 2022-06-17 02:01:59 +08:00
bunnie
87efc267f1 add PDDB backpressure items (#109 partial) 2022-06-17 02:01:30 +08:00
bunnie
66dc39c2a8 pddb sync does not require the handle to be mutable 2022-06-06 18:25:33 +08:00
bunnie
17446588bb fix read_all bug in PDDB 2022-06-05 01:45:05 +08:00
bunnie
21116a1693 fix another errant mut that wasn't needed in pddb 2022-06-01 00:06:55 +08:00
bunnie
bf535bd0e1 use interior mutability to handle callbacks
this eases a mutability requirement for the PDDB object
2022-05-31 23:54:16 +08:00
Sean Cross
3ae9d8abf0 apps/services: remove xous::xous_main macro usage
This macro will be removed, since it is no longer necessary. Remove this
macro across the project and replace it with a standard `main()`
function.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-05-31 13:33:30 +08:00
bunnie
1946d38a69 work around https://github.com/RustCrypto/AEADs/issues/410
We're trying to bring the AES version up so we can do CBC
(required for FIDO2) but unfortunately the aes-gcm-siv crate
is locked to an outdated version of AES, and this is blocking
on a type that is missing in the `block-ciphers` crate.

For now, we're using a patched version of aes-gcm-siv from
rozbb's repo, which allows us to move forward, but this needs
to be reverted once a new version of the official crate is patched.
2022-05-30 04:36:33 +08:00
bunnie
2cd4e80d9c bring PDDB to AES 0.8 compatibility 2022-05-30 03:08:41 +08:00
bunnie
e14e6206ed include a getrandom implementation that allows us to use modern randcore
A lot of packages are depending on 0.6 api, which requires a crate
that has OS specific TRNG calls. This patch keeps the 0.5 calls
inside the TRNG crate as direct API calls, and migrates to support
a 0.6 api to a patched version of getrandom that supports Xous.
2022-05-29 02:47:51 +08:00
bunnie
f4a18401e5 expose is_efuse_secured() API via the PDDB
is_efuse_secured() comes from rootkeys, but we don't want
to expose the full rootkeys API to the world.

PDDB already has an approved capability for accessing rootkeys,
so this single API is passed through via the PDDB object here,
allowing other procesess to query it.
2022-05-28 18:21:07 +08:00
bunnie
62cbc86aeb add automation strings 2022-05-24 23:21:26 +08:00
bunnie
8af8926ad1 make disk sizing errors non-fatal in hosted mode
it can be innocuous in the case that you're changing sizes
for debugging
2022-05-10 17:37:08 +08:00
bunnie
ea6cf27ed9 fix PDDB length field -- somehow this got flipped from the debug branch 2022-05-06 14:43:29 +08:00
bunnie
ee0f443883 hoist net thread exhaustion check to before we check the poll interval 2022-05-06 14:17:29 +08:00
nworbnhoj
0e561802ce enable notification modal text + qrcode
change the notification modal to accept the primary message and an
Option of independent text to display as a qrcode

pub fn show_notification(&self, notification: &str, qrtext:Option<&str>) -> Result<(), xous::Error> {}
2022-04-24 21:29:15 +10:00
bunnie
34c2ffe1a3 fix password entry to work with the new return type
also requires a call to `reset_action_payloads` for the
number of field elements to be recorded correctly!
2022-04-22 17:43:26 +08:00
bunnie
4ddd414ed6 fix modals API in the PDDB migration code 2022-04-22 15:55:15 +08:00
bunnie
2c3143f790 Merge branch 'main' into multi-text-modals 2022-04-22 15:54:18 +08:00
bunnie
38d6765586 fix remaining bugs in migration, tested and passing on CI + real hardware 2022-04-20 03:48:05 +08:00
bunnie
b3c89e5f09 add migration of secret basis for v1->v2; fix some interior mutability issues
Probably nobody is really using secret Bases right now, but it's
probably a good idea to lay out all the infrastructure and start
testing it for when it comes time to do it with data that really
matters.
2022-04-19 04:50:15 +08:00
bunnie
4ba34bbd58 intermediate work on v1->v2 migration, need to set aside to fix a libstd problem 2022-04-19 00:45:10 +08:00
bunnie
c8a38b4bf5 fix representation of va in pagetable so we're not burning 12 bits on 0's
this is a v2 fix, incompatible with v1
2022-04-19 00:44:45 +08:00
bunnie
7553053f29 v0.0.2.1 initial implementation - still needs migration code
Previous version was v0.0.1.1
New version is v0.0.2.1 (don't ask me why I skipped v0.0.2.0)

The most significant change is that the key derivation for the basis
is now split into two keys. One key is used for the page table ECB
encoding, and a different key is used for the data AES-GCM-SIV encoding.

This is because someone much smarter than me looked at my scheme
and said it is very dangerous to use the same key for two different
types of ciphers. This person is correct, so it had to be rectified.

For the System Basis, the keys are encrypted using the root keys
wrapper API. In this case, we simply create two random keys, one for
the page table, and one for the data region, and wrap them using
the root keys API.

For user secret Basis, the derived keys are sent through HKDF
with two different "info" strings to derive a pair of keys from
a common password.

In case the cryptographer who looked at this is reading along,
note that the basis name is encoded as a fixed-length, 0-padded
record, so, we don't encode the length of the basis in the hash
used to derive the keys. When I explained the scheme I thought
I was using a variable-length basis name to derive the key, but
when I checked the code, it's always 64 bytes long, and bytes
that are not used are 0-padded, so, I think we don't need the
length encoding field.
2022-04-18 15:59:15 +08:00
bunnie
cd8c762c55 patch up QR code PR, change method signature as discussed
now any modal notification can also render as a QR code
2022-04-08 15:19:56 +08:00
Gianguido Sorà
8123393a7d Merge branch 'main' into feature/multi-text-modals 2022-04-07 16:39:03 +02:00
bunnie
d49c35dc0c verify that unmount callbacks are working 2022-04-07 17:58:37 +08:00
bunnie
52d56f39f1 wire up PDDB key unmount callbacks (oops) 2022-04-07 17:37:25 +08:00
bunnie
6ac4ef292b add basis mount/unmount to pddb shell commands, fix some UX formatting 2022-04-07 15:32:47 +08:00
bunnie
1ad7c43f81 refactor pddb ux thread to use deferred response messaging 2022-04-07 14:46:44 +08:00
bunnie
92c6a93147 eliminate yield loop on blocking mount poller 2022-04-07 00:04:25 +08:00
bunnie
b75df1efb3 annotate rationale on multiple variants of pddb mount calls 2022-04-06 23:46:39 +08:00
Gianguido Sorà
83e61f5f1b feat: add wiring for multiple TextEntryPayloads drawing 2022-03-31 17:54:22 +02:00
bunnie
8b5ad2657b an aborted attempt at doing a clean-shutdown for hosted mode
This doesn't work because the shutdown call destroys all the servers then terminates
the processes. But, the problem is that the processes are still
running, but servers are disappearing. So what you end up with is a
panic when the first message gets delivered to a server that no longer
exists, and then you end up with the program hanging on that panic
because some servers haven't been destroyed/exited yet and are now
just spinning waiting to be scheduled. so the call works in the tidy
kernel test cases but to get it to work on a full OS you'd need to
have a way to put every server into an eternal blocking state before
making the shutdown call (to prevent new messages from being issued
while servers are going away), which is more work than i'm willing to
put into the problem right now.
2022-03-31 01:10:58 +08:00
bunnie
4f1f4a3907 optimize journal resolution code
We have the journals tracked in a HashSet, so, use the hashing
properties instead of an O(N) search.
2022-03-29 16:39:54 +08:00
bunnie
0bdf21b3ac retire debugging infos 2022-03-29 16:33:40 +08:00
bunnie
c542807958 fix PDDB bug where journal numbers on pages were incorrectly restored
An edge case was missed where a page which was allocated out of
the FSCB and then later written to disk was having its journal
number set to 0.

The root cause of the logic issue is that the page table itself
does not store a journal number. It's assumed that the page table
is ground truth, and is unjournaled.

As long as the pages don't exist in the FSCB, that's not a problem,
the journal number doesn't matter.

But, let's say this sequence happens:

1. FSCB allocates some free space
2. process takes that free space allocation and commits it to its own page table
3. FSCB update log gets a note that free space is now allocated
4. system reboots
5. process reads its page table from disk. There are no journals associated
with the page table on disk, so, it assigns a journal number of 0 by default.
6. we now have a conflict between the Process' view of the journal age
of that page table entry, and the FSCB's view.

The fix to this is during the page table scan, to check and see if
any candidate pages happen to already exist inside the FSCB. If it does,
then you adopt the latest journal number for that page in the FSCB.
This keeps the two views consistent.

A second bug was also identified while fixing this where the "Random"
order we were hoping for in visiting the free space cache as an
iterator is less random than we had hoped. So, there is a patch to
extract the free pages and pull one random one from the pool, using
entropy coming from the TRNG. This is less computationally efficient,
but at least it is correct and has more transparent properties
than relying on the indeterminism of the HashSet implementation.
2022-03-29 16:26:55 +08:00
bunnie
bdd194ea7b retire rust issue 90195 work-around in PDDB 2022-03-29 16:25:33 +08:00
bunnie
4eec54620c fix warnings and debug messages 2022-03-29 04:39:33 +08:00
bunnie
9a41b18400 fix bug where dictionaries with 1 entry fail to sync
also ensure that at least 1 byte is reserved for any key
that is allocated; this helps avoid edge cases dealing with
zero-length key entries later on.
2022-03-29 04:38:15 +08:00
bunnie
55b05cfd80 add debugging routines; fix zero-length key creation case
- add a "dangerous debug" command that only exists when the feature
"pddbtest" is enabled. The feature can't work on real hardware, but
still, it enables things like dumping of the database and storing
the keys for inspection, so it's isolated in a feature that prevents
the code from being compiled in the first place when it's not selected.

- fix a bug where keys that have zero length aren't being created
correctly. this is a common use case for reserving some space for
data that isn't yet available.
2022-03-29 01:47:28 +08:00
bunnie
18c202dcee start of testing code for the PDDB 2022-03-28 22:56:32 +08:00
bunnie
c69b9e451d unbreak hosted mode 2022-03-28 04:04:38 +08:00
bunnie
ae0b1f1288 fix bug in dict delete.
also soften a journal fault message to help with debugging.
that it no longer panics does not mean it is right...
2022-03-28 03:36:27 +08:00
bunnie
9555287273 remove a debug print 2022-03-27 17:07:48 +08:00
bunnie
727135b347 throttle debug messages 2022-03-27 16:44:26 +08:00
bunnie
fc4a598a9b add a command to reset RTC after a PDDB format 2022-03-27 16:44:13 +08:00
bunnie
515d49db08 add a sync operation after every write
I did observe some PDDB corruption that happened today that looks
ostensibly linked to a record that didn't sync up with the disk,
so, let's be safer and sync everything. We can work on optimizing
performance after correctness is guaranteed.
2022-03-27 02:55:19 +08:00
bunnie
a103536b58 add a non-blocking mount poller to the PDDB
The problem with the current is_mounted() call is that it blocks
while the PDDB is requesting a password or actively formatting
(otherwise it doesn't block). It's actually just...partially blocking.

That doesn't cut it for libstd-linked system services that need to run but
before the PDDB comes up. In particular, we need to be able to respond
to time queries *somehow*, even before our timezone and offsets are
loaded from the PDDB.

I considered doing a callback mechanism for mounting, but this
is kind of complex and probably not worth it. Instead, a separate
thread that can return the value of a single boolean revealing
the mount state of the PDDB is created, and you can create an
object that's capable of inspecting that boolean.
2022-03-27 02:52:29 +08:00
bunnie
91d0511fa4 add Seek trait to the PDDB 2022-03-26 02:34:57 +08:00
bunnie
0abd4a1db6 add a response code for dictionary not found 2022-03-25 23:50:26 +08:00
bunnie
92f1552d94 Merge branch 'main' into bunnie-dev 2022-03-24 03:32:36 +08:00
bunnie
0bcc93c78e attempt to add persistent storage to hosted mode PDDB
This will create a 100MiB file in tools/ppdb-images/hosted.bin
(sorry!!) which is persistent storage for hosted mode.

Delete the file if you want to start over from scratch.

It's...pretty brittle. I expect problems, but maybe it will at
least get things moving.
2022-03-24 03:21:00 +08:00
bunnie
bfb8a17cd5 strip out legacy RTC calls in preparation for refactor 2022-03-24 02:19:26 +08:00
bunnie
36ea69f6df add a one-liner API call to block execution until PDDB is mounted 2022-03-22 22:18:23 +08:00
bunnie
86e86df68c add a one-liner API call to block execution until PDDB is mounted 2022-03-22 22:16:46 +08:00
bunnie
9f6e444e3f back out testing structures 2022-03-10 23:45:49 +08:00
bunnie
e8b13fcb44 integrate TTS into more dialog boxes and modals
also tighten up the `blocking` method for TTS for prompts that
absolutely should not be truncated or overriden.
2022-03-10 23:40:35 +08:00