Commit graph

404 commits

Author SHA1 Message Date
bunnie
d331d405ee bump version numbers for publication 2022-10-25 17:23:40 +08:00
bunnie
e04d135e83 add vault app to sources 2022-10-24 00:36:51 +08:00
bunnie
f05a879d93 add a generic performance log analysis tool 2022-10-23 23:16:56 +08:00
bunnie
9489342d4b cleanup error messages from perflib 2022-10-23 23:15:18 +08:00
bunnie
805a7d528c flip back to using remote references 2022-10-20 16:30:06 +08:00
bunnie
86d2437f03 improve formatting so section alignments are easier to debug 2022-10-20 16:14:56 +08:00
Sean Cross
ff3f307a14 tools: read-tags: call output xous_presign.img
The use of the name `args.bin` is now deprecated.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-10-20 14:11:48 +08:00
bunnie
0a361cac20 bump version to 0.0.8 to asborb factory reset change 2022-10-19 04:47:55 +08:00
bunnie
76ed23010d erase staged gateware as part of --factory-reset 2022-10-19 04:44:04 +08:00
bunnie
8e05925cfa somehow the transition to local paths got missed in the merge 2022-10-18 00:39:01 +08:00
bunnie
0584ad2f2c switch to local paths to see if this helps the build 2022-10-18 00:26:56 +08:00
bunnie
4b368d71db bump versions of crates 2022-10-15 01:31:54 +08:00
bunnie
63a87ca1bc bump all the versions 2022-10-14 16:54:43 +08:00
bunnie
ff30131f43 absorb more fixes into the kernel 2022-10-13 04:08:56 +08:00
bunnie
3503965f98 bump versions to absorb xous-rs changes 2022-10-13 03:41:16 +08:00
bunnie
02b80425f5 bump xous-rs to absorb change in language specification 2022-10-10 03:07:44 +08:00
bunnie
9732ecdc18 add v0.9.10 to restore URL list 2022-10-09 23:37:48 +08:00
bunnie
01b68cbdc9 fix development roll-out process for picking a restore kernel 2022-10-09 14:18:57 +08:00
bunnie
99c6e005ad fix hosted mode 2022-10-09 02:46:24 +08:00
bunnie
6230c922ac fix backup/restore scripts 2022-10-09 02:36:48 +08:00
bunnie
8ac80f8c42 update tools to work with checksums (partial update) 2022-10-09 01:12:03 +08:00
bunnie
c33dee152d fix backup opcode setting 2022-10-08 22:20:50 +08:00
bunnie
c471ca0fbb fix overly restrictive version match on backup version 2022-10-08 15:29:37 +08:00
bunnie
a0791ff092 back out patch of the opcode change -- the backup script did that in error 2022-10-08 02:40:55 +08:00
bunnie
89d050891b fix incorrect opcode setting on backup 2022-10-08 02:40:07 +08:00
bunnie
44d3c19733 fix hash check computation and op offset 2022-10-08 02:26:00 +08:00
bunnie
42052fdcc5 fix padding on hash check in backup tools 2022-10-08 01:53:28 +08:00
bunnie
a416069002 add hash protection on the metadata header
stage is set for hash protection on the PDDB data itself.
that comes next.
2022-10-08 01:22:14 +08:00
bunnie
0f9195f305 add some sanity checks on the backup header
this tries to catch the cases that:
  - user tried to run the backup without first running "prepare backup..."
  - the USB connection is a little flakey and returning bum data

We can only inspect the plaintext portions of the header, which contains
things like protocol version numbers, time stamps, and language codes.

We use the timestamp validation heavily because it's the only field that
isn't almost always going to be a 0 or 1 when correct, which are highly
likely to be reported anyways if there's a problem with the USB link.

This does assume the user sets the time correctly on the device
doing the backup, and on the Precursor itself.
2022-10-06 00:53:52 +08:00
bunnie
3bca9de2a0 bump version numbers for xous-kernel patch
this fixes tests in CI
2022-10-05 23:51:35 +08:00
bunnie
969f93b001 cross-repo update versions from xous-rs bump 2022-10-05 18:32:44 +08:00
bunnie
776a47f4bc missed a file 2022-10-02 02:21:06 +08:00
bunnie
15bcb6328f fixup other refs 2022-09-27 03:09:41 +08:00
bunnie
3579dfdd17 update xous-ipc and xous versions 2022-09-26 16:07:35 +08:00
bunnie
796256cfb0 make usb_update.py appear only once in the tree 2022-09-19 12:44:55 +08:00
bunnie
5461a70be1 add some generated data for later review 2022-09-16 01:52:06 +08:00
bunnie
46a2d8609f add perflib modules
this library can be re-used across multiple scenarios that require
performance profiling
2022-09-16 01:48:59 +08:00
bunnie
3a893213ff update the generated images to the latest data 2022-09-16 01:47:30 +08:00
bunnie
e12f651ac4 update analysis framework to use the binary files output by the latest changes 2022-09-16 01:47:02 +08:00
bunnie
3e79e3e430 add ability to read out data based on a list of virtual sectors
This adds the `--dump` command, which requires a file that is
a copy/paste of a generated virtual to physical table. An example
of this log would look like this:

```
INFO:shellchat::cmds::net_cmd: V|P 6000d000 40799000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 6000e000 4079a000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 6000f000 4079b000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60010000 4079c000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60011000 4079d000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60012000 4079e000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60013000 4079f000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60014000 407a0000 (services\shellchat\src\cmds\net_cmd.rs:770)
INFO:shellchat::cmds::net_cmd: V|P 60015000 407a1000 (services\shellchat\src\cmds\net_cmd.rs:770)
```

and so forth. The script can parse this raw log to create the necessary
page table mappings to read a large region of virtual memory and save
it as a contiguous chunk in a file.
2022-09-16 01:39:49 +08:00
bunnie
57291a47f2 add some sample outputs from the analysis scripts 2022-09-11 21:16:45 +08:00
bunnie
088648409e add constant time analysis script 2022-09-11 19:13:38 +08:00
bunnie
2ee0d4dfce remind myself how to update the precursorupdater
the latest docs don't work for us, so, document our method here.
Just can't figure out the .toml method of describing the project...
2022-09-09 15:39:52 +08:00
bunnie
4b72492c3e fix bug #216 2022-09-09 15:07:31 +08:00
bunnie
ca3cc711a4 Re-add usb_update.py, but with modified help to guide users
It's a bit confusing to run the wrong tool but hopefully there
are ample help messages sprinkled in this to guide the average
user to the right location.
2022-08-16 13:57:36 +08:00
bunnie
5e455b60eb add debug to backalyzer to dump encrypted private key 2022-08-12 20:10:19 +08:00
bunnie
58a8264080 fix some bugs in printing errored-out sectors
also print the sector data that errors out -- might be a bit
much at the moment but it's very helpful for immmediately
visualizing a loss-of-power event related write corruption
2022-08-06 16:00:15 +08:00
bunnie
3a5e385b9a add arg to specify file (optionally) in backup 2022-07-29 00:24:40 +08:00
bunnie
a68eb7cc73 add missing files for backalyzer; add PTE conflict resolution 2022-07-28 04:47:02 +08:00
bunnie
e2f7a75f87 improve help message 2022-07-26 03:43:31 +08:00
bunnie
6266101727 backalyzer now works with PDDB in hosted mode.
Specify file as `-f pddb-images/hosted.bin` and use `--hosted`
2022-07-26 03:42:11 +08:00
bunnie
c889f5576f cleanup backalyzer. i think it's basically ready to go now 2022-07-26 03:09:09 +08:00
bunnie
22a85ba022 working draft of backalyzer 2022-07-26 02:53:19 +08:00
bunnie
ba7532a4fe fix backup length
we were accidentally also backing up the WF200 firmware as well,
chop that off so that the analysis scripts & restore works correctly.
2022-07-26 02:52:40 +08:00
bunnie
ef29e6e504 re-org the python pddb analysis scripts
allow for elements of pddbdbg.py to be shared with the backalyzer.py
2022-07-26 00:06:16 +08:00
bunnie
494ec7c2b3 improve the basis name validator - ':' is an illegal character in a basis name. 2022-07-26 00:05:45 +08:00
bunnie
a0a51cab5c emphasize that you must reboot by using the paperclip 2022-07-23 16:23:05 +08:00
bunnie
97b13e7df3 move the OG signing tool to legacy status 2022-07-23 16:08:47 +08:00
bunnie
b4a784391d clean up the tools directory 2022-07-23 16:07:33 +08:00
bunnie
c43aa7f77e remove egg and dist from the index as it is dynamically generated 2022-07-23 00:11:32 +08:00
bunnie
a43e0640f9 i'm not sure if these typically end up in repos, but they're there... 2022-07-22 22:29:32 +08:00
bunnie
5bce709d3c some cosmetic improvements and release notes acknowledgements 2022-07-22 22:28:46 +08:00
Jeremy O'Brien
b47fccde6a precursorupdater python package improvements
move precursorupdater's main into its own namespace and handle both
running from the repo directly and from pip
2022-07-22 09:53:04 -04:00
Jeremy O'Brien
52bcaa58b2 small fixes to make the precursor updater python package work through pip 2022-07-22 09:41:43 -04:00
bunnie
bc758749fb try to explicitly name the package.
this still did not work.
2022-07-22 20:11:57 +08:00
bunnie
11402ec5e6 reorg again 2022-07-22 19:54:06 +08:00
bunnie
80a8bba268 try to re-org again to get this to run as a script 2022-07-22 19:53:46 +08:00
bunnie
237ed14d28 finalize renaming so we don't have to deal with _ vs - ambiguities 2022-07-22 19:06:13 +08:00
bunnie
ee9f815fd6 restructure project 2022-07-22 19:05:26 +08:00
bunnie
4fa1236ce6 remove the _. i don't know the rules for how this transforms to - and when. 2022-07-22 18:54:56 +08:00
bunnie
aed8c89373 more restructer attemptings 2022-07-22 18:54:40 +08:00
bunnie
4e1a3beb3c try to restructure repo to work with...idk why i am even trying. 2022-07-22 18:54:21 +08:00
bunnie
ee4e2e5d65 fix metadata on packaging for precursor_updater
still not sure how to make this actually work as i want it to,
but at least there's a thing you can run to like...get dependencies,
I guess.

lame.
2022-07-22 18:10:32 +08:00
bunnie
8f6fa5f53d add a first draft of precursor_updater.py
this should be an "all in one" updater.
2022-07-22 03:46:31 +08:00
bunnie
bad4865ea3 shift restore.py to look at the tagged releases for artifacts 2022-07-21 01:10:52 +08:00
bunnie
8dc9dddfdd modify pdddbg.py to handle swapped up DNA for verifying rekey ops 2022-07-19 03:17:32 +08:00
bunnie
8f439f8d61 add the beginning of a CI/analysis tool for backups
for now all the tool can do is just make sure that the
header is valid, but it can't decrypt or operate on
the backup itself.
2022-07-17 22:49:03 +08:00
bunnie
8e6728d743 improve error handling in restore script 2022-07-17 22:48:03 +08:00
bunnie
13e13fdfd6 add some explanation in the restore process 2022-07-17 18:50:28 +08:00
bunnie
802decb55a initial drafts at the backup and restore helper scripts
very, very rough but i was able to run through an entire
cycle exactly once successfully.

much more testing needed.
2022-07-17 03:13:42 +08:00
bunnie
9c7b1e3228 update update script verbiage to reflect the latest flow 2022-07-17 03:13:13 +08:00
bunnie
7248f47ca8 fix self-signing to reflect extra data for semvers
the loader is a little funky, and requires some maintainer hand-holding
due to the split of the page ownership between the root-keys
crate and the graphics-server crate.

Improved docs so I don't forget this detail in the future.
2022-07-11 16:11:32 +08:00
bunnie
2ad7ed10d6 bump semver version 2022-07-11 04:13:19 +08:00
bunnie
1256fcb411 add min version requirement and current version to kernel and loader 2022-07-08 18:56:05 +08:00
bunnie
40b16b3cdb inject current semver info into signature appendix on loader and kernel
this allows for a quick readout of the version of each of these
artifacts, inside a signed region, by just inspecting the last
few words of the binary object.
2022-07-08 04:08:37 +08:00
bunnie
ae8fe799bc add a fuzzer for basis lock/unlocking
this commits a script that will repeatedly rerun the `btest`
which creates several Bases, fills them with pseudorandom datat,
and goes through a lock/unlock cycle.

should be run with the disk size set to 4MiB for maximum stress
testing (the larger disk won't trigger an FSCB refill because its
hard to run out free space on the larger disk size)
2022-07-04 17:53:29 +08:00
bunnie
97ae56915e fix edge case in --dump mode for pddbdb.py
shouldn't crash when not in CI analysis mode
2022-07-04 17:52:59 +08:00
bunnie
f14cce55ff update the updater readme
The FPGA gateware region is blocked from trivial readout via USB:
this makes it a tiny bit harder to obtain the gateware image, which
contains the secret keys. This slightly aids users who haven't
burned any efuse keys and are using the default "null" key from
a trivial readout attack, and it also hampers a WBSTAR exploit.

This makes the most convenient path of attack to be loading
a custom xous.img (or otherwise exploiting the kernel) with
a gadget that is capable of reading the gateware region
through priveleged code, and then relaying it over USB through
some run-around of the firewall. This primitive isn't impossible
to create but it doesn't exist and has no use in a normal
operational flow, so, it's going to at least force an attacker
to write some code, as opposed to using off the shelf utilities
we provide.
2022-06-29 13:20:52 +08:00
bunnie
e32ea4d3f5 fix #174 2022-06-10 21:36:53 +08:00
Sean Cross
aa436cbcc7 tools: elf: don't get raw_data for no_copy sections
The `xmas-elf` crate doesn't deal with `raw_data()` of sections that are
`NO_COPY`. These sections may point to data that is outside of the file.

In order to avoid hitting this assert, don't call `raw_data()` on
sections that are marked `NO_COPY`.

Signed-off-by: Sean Cross <sean@xobs.io>
2022-06-10 16:52:57 +08:00
bunnie
7112b6a15f add a script to help extract keyboard mappings for unknown keyboards
use `usb kbdtest` to output a mapping assuming US-101 maps
it will then derive an appropriate keymap based on that mapping

doesn't support non-ascii characters (yet)
2022-05-17 16:39:48 +08:00
bunnie
02daea2f0e add a note for STAGING area as well 2022-04-22 02:48:52 +08:00
bunnie
5bdd3a9f51 modifications to prepare for enhanced USB security
The next release will feature an access control on USB so that
root keys and the encrypted gateware cannot be read back and
transitively the writes would be guttered (but this is not a problem
because the write target address is not set by the memory location
of the write, it's actually set by the location specified in
the program command to the FLASH).

This means that the updating program has to be modified accordingly;
now, for writes, a single location is targeted (and as alluded above,
this is funneled to a write FIFO buffer, and then the actual address is
set with the page program command); and for readback, attempts to read
back the SoC area are flagged and noted as impossible due to security
precautions.
2022-04-22 02:48:43 +08:00
bunnie
b39f4c38eb fix update scripts so updates don't wipe root keys :-O 2022-04-20 17:15:46 +08:00
bunnie
9878dfc1a9 add a note about resetting the device properly after an update 2022-04-20 17:03:02 +08:00
bunnie
a31030bf90 remove excess wait state and clarify wait text 2022-04-20 16:59:19 +08:00
bunnie
15105cd0b2 fix update scripts to prevent device from self-resetting during updates 2022-04-20 16:57:10 +08:00
bunnie
4f9f1116fc fix various bugs in pddbg
note that there is still pddbg_v1 for the older version of the pddb
2022-04-20 03:47:17 +08:00
bunnie
758ed1ba31 some more fixes to pddbg to bring it up to v2. now passing CI. 2022-04-19 23:28:47 +08:00
bunnie
2d0af08266 bring pddbdbg up to v2 2022-04-19 15:57:51 +08:00
bunnie
d0f632965e add the old version of pddbg for debugging v1.x databases 2022-04-19 04:49:40 +08:00
bunnie
ba3639b00a prepare for v2 migration. more edits coming 2022-04-19 00:44:30 +08:00
bunnie
52f1afa780 commit some improvements to the CI framework 2022-03-31 00:28:49 +08:00
bunnie
e4eed87a55 Merge remote-tracking branch 'origin/main' into bunnie-dev 2022-03-29 16:45:56 +08:00
Aaron Bieber
6f98940189 Switch to using env to find the path to bash.
Systems like OpenBSD and NixOS have bash in different locations. This
allows the various scripts to be run on those systems.
2022-03-28 16:43:47 -06:00
bunnie
afb9974309 issue a warning if dup journal entries are found (pdddbg) 2022-03-29 04:37:35 +08:00
bunnie
0cf27be3b0 make the partition on dump vs ci analysis cleaner 2022-03-29 01:53:42 +08:00
bunnie
23041ab1f1 add a dump mode to pddbdbg.py so we can just view records
helpful for manual debugging
2022-03-29 01:47:00 +08:00
bunnie
c79afd4220 make sure the directory is not deleted 2022-03-28 04:09:57 +08:00
bunnie
6cbc93f8de make sure the directory is not deleted 2022-03-28 04:09:19 +08:00
bunnie
526a7374b0 re-introduce PDDB structure nuking on SoC reset 2022-03-20 15:25:14 +08:00
bunnie
b52b73ddfa add functionality to override the CSR data in case it is corrupted 2022-03-19 16:56:03 +08:00
bunnie
b06d7c4bb7 add a note about the hard reset 2022-03-19 16:31:37 +08:00
bunnie
1856a83e3e fix ordering of updates to prevent mid-update resets 2022-03-19 16:22:09 +08:00
bunnie
2aa72fae18
Update factory_reset.sh 2022-03-19 09:01:40 +08:00
bunnie
d9f4f6d041
Update factory_reset.ps1
move soc update to top so we don't get resets from latest xous
2022-03-19 09:00:58 +08:00
bunnie
eae61df786
Update factory_reset.sh
move soc update to the top to avoid wdt resets
2022-03-19 08:58:41 +08:00
bunnie
f7d42f6c1b correct usb_update.py force message to the correct version offsets 2022-03-18 17:39:45 +08:00
bunnie
dc2c393c5c update scripts to accept a locale argument 2022-03-10 16:22:48 +08:00
bunnie
dd017fb291 retire file-based versioning to symlink methodology 2022-02-28 16:06:48 +08:00
bunnie
2b3178be82 add a powershell script for bleeding edge CI
Because i don't know how to do variables in powershell :-/
2022-02-28 14:07:42 +08:00
bunnie
65499eb0fc bump version code in the pddbdbg.py script as well 2022-02-28 04:16:06 +08:00
bunnie
0b454cb7a8 evict salamanders from AES-GCM-SIV in the PDDB
So you'd like to think that an authenticating cipher would like,
authenticate stuff on decryption. And AES-GCM-SIV was billed as sort
of AES with the pointy edges taken off.

Well, there's always one more bug, actually this time a salamander:
https://eprint.iacr.org/2020/1456.pdf Apparently one can forge
authentication if the attacker can control keys and ciphertext. In the
case of the PDDB I think the attack is largely theoretical, because
you'd have to forge through both the AES-GCM-SIV and the rather
insufficient nonce/murmur hash check on the PTE that locates the page
to be decrypted.

But it's never good to have cryptographic properties depend upon
something that's known to be insufficient, so let's assume the attacker
can simply forge keys and stick some data on the device.

This wouldn't be a problem for confidentiality but it could lead to
the ability to plant evidence, but more importantly could lead to code
paths that were thought to be protected against fuzzing being fuzzed
as you could conceivably create a Basis root record that "checked out"
and then the larger body of the PDDB code would try to chomp on other
pages that are forged, leading to a bigger attack surface. Plus who
knows what else might exist beyond salamanders...

This patch does a "key commitment" on the Basis root page, so that it
is now cryptographically hard to create a salamander.  Basically
instead of using the system root key directly in AES-GCM-SIV, the key
goes through a KDF that expands it with yet another nonce plus a bit
of header data. The KDF is SHA512/256 and it expands it twice,
changing one bit in the header data (as prescribed by the paper cited
above), to create a version of the key that is written to disk (the
commitment), and a version of the key that is used to
decrypt/authenticate the AES-GCM-SIV data. Only if both the commitment
and the authentication check out, do we accept a Basis root page as
valid.

I had some concerns about the slightly different structure of the
Basis Root page being detectable, but I have been assured that the key
commitment form and the MAC in AES-GCM-SIV satisfy a property known as
IND$-CPA ("indistinguishable from uniform randomness by a
chosen-plaintext attacker") by Rogaway[1], and, more strongly, "Secure
nonce misuse-resistant authenticated encryption (mrAE)" by Gueron and
Lindell (see appendix for def)[2]. Still, just out of total paranoia
and nothing else, I organize the basis root page such that the MAC is
in the identical location as the other non-committed pages...just in
case there is some mathematical property of the MAC that could be
leveraged to leak info about the existence of the basis root record.
The commitment nonce and sha512/256 result I understand better and
feel pretty comfortable that they are going to be strongly random and
indistinguishable from AES ciphertext.

Thanks to Michael Rosenberg for taking the time to explain all this to
me and to guide me through the process of making the fixes. It is a
major PDDB API-breaking change and I'm glad it's able to be rolled out
before most users have received a device.

[1] https://web.cs.ucdavis.edu/~rogaway/papers/ad.pdf
[2] https://eprint.iacr.org/2015/102
2022-02-28 03:13:26 +08:00
taivlam
152abdac6b
Fix minor typos
Fixed minor typos in comments (I guessed "ts" was "its")
2022-02-14 19:45:12 -05:00
bunnie
b6b620b9b0 fix some paths 2022-02-10 00:46:20 +08:00
bunnie
2c3425d3fa add a release updater for windows as well 2022-02-10 00:46:09 +08:00
bunnie
ec64782f32 remove a trailing space 2022-02-10 00:39:50 +08:00
bunnie
d88d552260 add a windows version of the factory-reset command 2022-02-10 00:39:38 +08:00
bunnie
fcb67d5d16 fix argument order 2022-02-04 04:25:58 +08:00
bunnie
ad8ba37cf9 add a script that does a factory reset
for those just getting started and who want a fast-path to update.
2022-02-04 04:18:30 +08:00
bunnie
11c88b170e implement a basis for anti-rollback upgrades
the actual transition to a new version isn't here. But,
at the very least, the root keys are now derived by hashing
them MAX_ROLLBACK_LIMIT times, which allows us to create a
key chain based on a global version number that can be incremented
while allowing derviation of older keys without compromising newer
keys.
2022-02-04 01:33:04 +08:00
bunnie
b3788a54af clarify the dependency policy 2022-01-10 16:50:10 +08:00
bunnie
995915ed4e nail down a couple more dependencies, and add a policy note to Cargo.toml
Each Cargo.toml now includes a comment which remnids maintainers
of the dependency policy for Xous (that is, nail down every dependency
to the minor number).
2022-01-10 15:54:27 +08:00
bunnie
fec180890b nail down dependencies in the related build toolchain flow
these are secondary attack surfaces -- code that isn't directly
in the kernel, but involved in the manipulation, handling, grooming
and preparation of source and binary files
2022-01-10 15:49:30 +08:00
bunnie
dd943c5416 pin down versions and harmonize versions across servers
Versions are pinned down to the minor number. Yes, this means that
we don't automatically pull in features and bug updates from crates.io,
but it also means we don't pull in new attack surfaces and code that
hasn't been tested or reviewed. I'd like this process to be very deliberate
and not automatic for Xous.

The `rand` crate is almost pinned down to a single version, except
for an odd dependency that's pulled in by hosted mode that depends
on rand 0.8.4 via a subordinate dependency via the `tempfile` crate.
Pondering if it's worth it to try and eliminate that, because the
0.8.4 version doesn't get compiled into any version that touches
real hardware. It just makes it harder to analyze the Cargo.lock
file, mostly. I think I'll leave it as-is.
2022-01-10 15:39:55 +08:00
bunnie
9a34de0065 rework flash memory locs; add command for re-creating a factory new device 2022-01-09 22:33:36 +08:00
bunnie
0c71622740 fix alignment bug in analysis code 2022-01-06 23:15:24 +08:00
bunnie
4c927f2c99 fix name records so they encode a length
This is one of the truly redeeming features of Rust is that a
major refactor of a core structure like this doesn't leave me
shaking in my boots -- I feel pretty good that this works
since everything passes the CI tests.

The original FS implementation did the "stupid C thing" which
just creates a fixed lenth field and uses null-terminated C
strings. Which actually restricts PDDB entry names to be ascii
as valid UTF-8 can encode single-byte nulls.

The intention was for names to be fully utf-8 compliant, but in
order to do that we have to burn one byte of potential filename
length and encode an overall length field for the name strings.

This change does that, which both makes the code a bit safer
at the expense of losing a character off of the maximum filename
length.
2022-01-06 18:53:01 +08:00
bunnie
d87920a772 add argument to erase the entire PDDB region. 2022-01-04 11:12:30 +08:00
bunnie
630dd6a784 make the CI options command line driven 2022-01-02 16:41:23 +08:00
bunnie
421a3814d8 clean up some warnings before moving on to further integration 2022-01-02 15:07:48 +08:00
bunnie
33b0593f9a fx fspace cache entry space check issue
was counting total number of pages, not actual free pages.
2022-01-02 01:41:55 +08:00
bunnie
fef355423b fix CI red herrings; add more debug granularty 2022-01-01 15:14:44 +08:00
bunnie
c02a8933ef speed up the analysis tool so it's a bit more practical to use in CI 2022-01-01 14:14:09 +08:00
bunnie
656f17f621 add extensive CI test 2021-12-29 05:49:07 +08:00
bunnie
6990513278 fixed a subtle bug in the pte sync algorithm
was mixing deletions and updates and because they are in arbitrary
order due to the hashset structure, sometimes i'd delete an entry
after it was updated.

this is now found and fixed.
2021-12-28 06:47:33 +08:00
bunnie
713c0d6e73 add a multi-iteration CI loop around the pddb test
this allows us to capture seeds and corners of interest
2021-12-27 16:15:51 +08:00
bunnie
4f7372bbd4 add log levels to the debug tool 2021-12-27 13:07:01 +08:00
bunnie
c15841cfdc test framework in place to look at multiple basis
found one bug already, need to refactor how PT encryptor is
handled
2021-11-28 23:54:58 +08:00
bunnie
af9f4cbde3 back out changes for stress testing the fastspace mechanims 2021-11-27 21:23:42 +08:00
bunnie
185d629b98 stress test fastspace allocation and regeneration
- confirmed that log entries regenerate on trigger
- confirmed that fastspace cache regenerates and contains a
new subset of blank pages.
2021-11-27 21:17:41 +08:00
bunnie
93ee050079 fastspace regen debugging -- passes test, but needs optimization
should be able to get away with "just" regenerating the fast space
log only if that's what we've run out of; however, doing that optimization
we run into out of space errors. maybe we need some tuning of the thresholds
for when to reclaim space
2021-11-27 02:19:57 +08:00
bunnie
e02df18374 fix reporting error in directory listings 2021-11-26 03:36:36 +08:00
bunnie
468e372bfa most tests passing, also, be a little smarter about cache fills. 2021-11-26 03:08:08 +08:00
bunnie
ca485d5541 fix off-by-one in small pool to dictionary storage mappings
consolidate index math into helper functions to reduce errors.
2021-11-25 03:14:29 +08:00
bunnie
8de0d70f43 debugging the patch-extend test.
probably should make a bunch of unit tests for the free dictionary
key tracking logic, it's too tricky to test out.
2021-11-24 03:57:50 +08:00
bunnie
66a5b97705 revise pagetable entry to be 7 bytes wide -- this will allow
us to have a full 64-bit address space upon expansion of the page
number (turns out this is probably important because of the crappy
increment-only allocator used for the large blocks)
2021-11-24 01:49:22 +08:00
bunnie
42e926fcab fixes for patch update test
lots of corner cases found and ifxed
2021-11-23 23:33:06 +08:00
bunnie
d716fea17a now passing dictionary add/remove test 2021-11-23 02:57:13 +08:00
bunnie
77d3f8a646 fix bug in python aes-gcm-siv reference implementation 2021-11-23 02:56:54 +08:00
bunnie
eda60a5d9c base case test done, with CI checking 2021-11-22 18:26:19 +08:00
bunnie
62059a9f93 relocate images to a directory 2021-11-22 14:00:43 +08:00
bunnie
1874c8e72d most of the core routines are now in place
can create/delete dicts, create/delete/update keys.
basis deletion is...just forgetting the password. that's the point,
at least. could maybe extend the basis AAD to include some small
amount of derived data that could be deleted to facilitate basis
deletion...

anyways. next up is...lots and lots of testing.
2021-11-22 04:46:36 +08:00
bunnie
193f563782 very basic small key additions working as expected now 2021-11-21 02:21:25 +08:00
bunnie
c62b2a25fb dictionary adding now seems to work 2021-11-18 02:01:30 +08:00
bunnie
507d99ee41 moving toward refactoring the basis sync function 2021-11-18 00:09:55 +08:00
bunnie
bbbaa6b810 clean up the structural definition of the dictionaries and keys 2021-11-16 18:45:24 +08:00
bunnie
49d00c783e basic mounting of the PDDB & readout of system basis 2021-11-16 02:58:33 +08:00
bunnie
d7cc518552 add mbbb mechanism 2021-11-15 19:05:03 +08:00
bunnie
65bbc82ef5 commit to Repr(C), include doc comment for rationale and tests 2021-11-15 16:38:57 +08:00
bunnie
ce51205155 checkpoint with memory placement and disk storage
this probably isn't new to most other people but this has been
a very visceral exercise of how tricky it is to use Repr(c)
for data layouts in anything but the most trivial data structures.

Unfortunately the Basis root data structure needs to be fairly rich,
so we're playing around with using rkyv to help with the serialization
to disk but I'm also not convinced this is a good idea yet. This
checkpoints commits a "working" rkyv implementation, but I'm not
entirely happy with the notion that rkyv can shuffle around data
bits based on its particular version or decisions in the serializer
crate. I'd like deterministic, repeatable serialization that will
be solid through lots of revs...
2021-11-15 16:00:01 +08:00
bunnie
ba9b136f87 debugging alignment issues on serialization of compound structures 2021-11-15 04:03:55 +08:00
bunnie
94a86c542d add an entrails parser for the PDDB 2021-11-14 19:02:34 +08:00
bunnie
8b59bcdf61 add indices to create_image output
makes it easier to figure out what IniE section corresponds
with which PID.
2021-11-06 21:03:28 +08:00
bunnie
c87acaeddc clarify a bit of rounding code in the create-image routine 2021-11-06 20:30:01 +08:00
bunnie
6360d1c902 tighten up the hard-coded locations for the trng test
these locations look good for a while, now that i see the
structure of data allocation (it's split between top and bottom)
2021-10-31 02:58:43 +08:00
bunnie
5f327126ab Merge branch 'bunnie-dev' of github.com:betrusted-io/xous-core into bunnie-dev 2021-10-30 03:16:13 +08:00
bunnie
72df2ff7e4 commit tester stability tweaks 2021-10-30 03:15:53 +08:00
bunnie
ad66460543 move the trng tester ram locations on the corresponding host-side test script 2021-10-30 03:14:33 +08:00
bunnie
73c131cbcc fix help message 2021-10-07 01:09:53 +08:00
bunnie
42e298d79b add a sleep between update commands 2021-10-05 22:34:53 +08:00
bunnie
792ad953eb add wf200 artifacts to the update script 2021-10-05 22:28:30 +08:00
bunnie
2e40546a88 add a force overwrite option to the updater for the soc mode 2021-10-04 16:30:41 +08:00
bunnie
0f00e36dea loader was never updated. oops. 2021-10-04 16:26:33 +08:00
bunnie
6b0b768081 add a line break on the exit to call out the reminder 2021-10-04 15:48:37 +08:00
bunnie
e3ff753e83 add a line break on the help message 2021-10-04 15:48:23 +08:00
bunnie
9994b38e16 clarify update_ci role with respect to the SOC image 2021-10-04 15:47:02 +08:00
bunnie
7bfdce147c modify the update_ci script to be aware of the release tree 2021-10-04 15:38:04 +08:00
bunnie
966615270f update trng test wdt ping method 2021-09-09 22:38:51 +08:00
bunnie
f3786b01a1 Merge branch 'main' of github.com:betrusted-io/xous-core into main 2021-09-07 19:01:02 +08:00
bunnie
840cccecda make the filename more obvious for the purpose of the legacy update script 2021-09-07 19:00:34 +08:00
bunnie
da9078749f add a script for xous 0.8 devices. because there are still a couple out there 2021-09-07 18:59:47 +08:00
bunnie
e510525a0a move wycheproof-import to tools 2021-09-06 14:42:19 +08:00
bunnie
e57587d9c8 update documentation to reflect bitstream padding added to align AES blocks 2021-09-05 18:31:42 +08:00
bunnie
0ae2d13602 update trng_test.py to xous 0.9 2021-09-05 17:47:52 +08:00
bunnie
d20a06e4eb fix update script to use the correct precursor 2021-08-30 03:30:45 +08:00
bunnie
69d9e38c8b add EC updates to the CI update script 2021-08-24 23:22:41 +08:00
bunnie
33d14b6ee3 add some timeouts after each phase, for the device to reboot 2021-08-24 22:59:10 +08:00
bunnie
1af98f7e43 Merge branch 'main' of github.com:betrusted-io/xous-core into bunnie-dev 2021-08-24 22:39:50 +08:00
bunnie
99ff36f529 add a quick CI update script 2021-08-24 22:37:31 +08:00