Commit graph

10340 commits

Author SHA1 Message Date
bunnie
656f17f621 add extensive CI test 2021-12-29 05:49:07 +08:00
bunnie
6990513278 fixed a subtle bug in the pte sync algorithm
was mixing deletions and updates and because they are in arbitrary
order due to the hashset structure, sometimes i'd delete an entry
after it was updated.

this is now found and fixed.
2021-12-28 06:47:33 +08:00
bunnie
713c0d6e73 add a multi-iteration CI loop around the pddb test
this allows us to capture seeds and corners of interest
2021-12-27 16:15:51 +08:00
bunnie
4f7372bbd4 add log levels to the debug tool 2021-12-27 13:07:01 +08:00
bunnie
afd1086cda commit unsaved changes meant to be part of previous commit 2021-12-27 12:32:18 +08:00
bunnie
1fab068296 add seed setting from the XOUS_SEED environment variable 2021-12-27 12:29:29 +08:00
bunnie
dd14dfc4c6 retrofit the seed into an atomicu64 so we can set it elsewhere 2021-12-27 02:49:08 +08:00
bunnie
809dbbdbc2 fix a bug in the reserved amount of data in large keys 2021-12-26 16:29:35 +08:00
bunnie
0bd110c163 determinism achieved for testing. now to squash some bugs. 2021-12-26 01:50:30 +08:00
bunnie
37b46fae98 make determinism an optional feature 2021-12-26 01:42:36 +08:00
bunnie
d4027fa8a2 sort all the lists in the test code so the vectors are deterministic
not converting core structures from HashSet to BTreeSet though --
the idea is that in production the mappings should be fully random,
but we are just trying to get reproducibility in hosted mode so
that we can debug some hairy edge cases more easily.
2021-12-25 16:44:24 +08:00
bunnie
66606ed3e0 initial attempt to make hosted mode more deterministic
replace all threadRng with a seeded Rng
2021-12-25 15:58:27 +08:00
bunnie
c1c57edfd6 some mods to the logging infra to chase down a "rare" issue 2021-12-22 07:09:30 +08:00
bunnie
5d226a002a prepare to start using the modals server for the pddb 2021-12-17 03:11:06 +08:00
bunnie
c027b617b6 update some docs to refer to the modals refactor 2021-12-17 02:59:18 +08:00
bunnie
8c66e04bc3 refactor RTC module to use modals server 2021-12-17 02:49:38 +08:00
bunnie
725d943198 add interlocking so multiple threads don't interfere with each other 2021-12-17 01:55:36 +08:00
bunnie
0fb1bea428 add radio buttons and checkboxes to the modals meta-handler 2021-12-17 00:32:44 +08:00
bunnie
91836e5574 add a warning about the fixed size of the IPC payload for selected checkbox items 2021-12-16 23:21:54 +08:00
bunnie
44453bcbde refactor checkboxes and radiobuttons to use dynamically allocated lists 2021-12-16 23:16:21 +08:00
bunnie
60fc03351a add progress bar primitives 2021-12-16 21:39:49 +08:00
bunnie
4a0f062558 add notification api 2021-12-16 02:28:37 +08:00
bunnie
dcad15284d refactor tests into a separate file 2021-12-16 01:48:11 +08:00
bunnie
2ddbc224b2 check in to swap branches 2021-12-15 14:08:38 +08:00
bunnie
f3f3e677d2 turn the validator error message into a newtype
this allows us to fine-tune the error message static size parameter
later on more easily
2021-12-15 13:53:53 +08:00
bunnie
e48c472b32 helps to add the code for the new service 2021-12-15 13:46:43 +08:00
bunnie
60d6d08933 basic framework for modal API wrapping in place 2021-12-15 13:46:24 +08:00
bunnie
f62eca9b88
Merge pull request #92 from hkjn/20211214-document-deps
Document dependencies on libssl-dev libxkbcommon-dev, newer rustc
2021-12-15 03:03:34 +08:00
Lairs Johnstone
eb26ae5a32
Document dependencies on libssl-dev libxkbcommon-dev, newer rustc
Without these two packages, `cargo xtask run` failed for me on Debian 11 (bullseye).

The version of `cargo` and `rustc` provided by the apt package `rustc` also threw the error mentioned about `feature resolver is required`. With the install script from https://rustup.rs a much newer version was provided, `rustc 1.57.0 (f1edd0429 2021-11-29)`, compared to [`rustc (1.48.0+dfsg1-2`)](https://packages.debian.org/bullseye/rustc) provided by `apt install rustc`.

Signed-off-by: Lairs Johnstone <me@hkjn.me>
2021-12-14 18:57:52 +01:00
bunnie
c852f5fbbc light refactor to use HashMap instead of linearly searched array
GAM predates `std` so we had a linear array for storing UX
contexts, that had to be manually grown every time we added
an element. Convert that to a Heap-allocated structure.
2021-12-14 13:47:36 +08:00
bunnie
de4e84abf5 starting in on the larger system integration of the PDDB
Trying to find a new approach to making dialog boxes that
are cleaner and don't require as much code repetition. Probably
about to go on a long tangent.
2021-12-13 14:24:14 +08:00
bunnie
48392ecd13
add margin on the ec reset time
4s seemed like a long time but i found one board that can't quite finish reset in that time, about 70% of the time it needs just a half second longer.
open up the window to 6s, just to ensure we don't get any false negatives.
2021-12-02 10:53:39 +08:00
bunnie
c84ed61ecd basis overlay test done & passing
the formatting of the listing output is a bit messed up but
i'm too tired and too late to deal with that. at least the
core logic seems functionally correct!
2021-11-30 00:38:29 +08:00
bunnie
8066b8fb5e fixed the bug with the pt map encryption, now we hvae two basis!
more testing needed, but looking good so far.
2021-11-29 00:25:09 +08:00
bunnie
c15841cfdc test framework in place to look at multiple basis
found one bug already, need to refactor how PT encryptor is
handled
2021-11-28 23:54:58 +08:00
bunnie
5a12e83ff2 add first cut at basis key derivation function 2021-11-28 20:42:26 +08:00
bunnie
af9f4cbde3 back out changes for stress testing the fastspace mechanims 2021-11-27 21:23:42 +08:00
bunnie
185d629b98 stress test fastspace allocation and regeneration
- confirmed that log entries regenerate on trigger
- confirmed that fastspace cache regenerates and contains a
new subset of blank pages.
2021-11-27 21:17:41 +08:00
bunnie
93ee050079 fastspace regen debugging -- passes test, but needs optimization
should be able to get away with "just" regenerating the fast space
log only if that's what we've run out of; however, doing that optimization
we run into out of space errors. maybe we need some tuning of the thresholds
for when to reclaim space
2021-11-27 02:19:57 +08:00
bunnie
e02df18374 fix reporting error in directory listings 2021-11-26 03:36:36 +08:00
bunnie
468e372bfa most tests passing, also, be a little smarter about cache fills. 2021-11-26 03:08:08 +08:00
bunnie
0df5f1c926 fix key de-allocation bug
forgot to reduce key count for the dictionary record when
a key is removed.
2021-11-25 22:03:37 +08:00
bunnie
85e48c80ed small keys had their data missing because i deleted it :-/
it's one of those bugs where you're like oh...right. it's doing
exactly what i told it to. I just told it to do the wrong thing.

Right. If you're updating data at the end of a file, you kind of
don't want to delete the file entirely then do the update.

Fix the code so that in the case of a file extension of a small
key, you're re-inserting the key in its entirety (not just the
updated portion) on the extension case.

Also handled an issue with large files where key extension
should not lead to overwrite of the neighboring data, and put
a cap of the large file limit of 32GiB. See comments for thoughts
on the trade-off of the large file limit versus the allocation
limit of the number of files and the physically realistic
outcomes for the PDDB. Basically as written the PDDB can handle
32GiB files (big enough to hold a blu-ray movie, but not a 4k/8k
movie, uncompressed movies, and/or large virtual machine images),
and can handle creation of files at an average rate of one per
second for 10 years without stop before running out of virtual
memory space.

This isn't "Web scale" as large servers will
definitely require files bigger than 32GiB these days, and
can easily create files at a rate of thousands per second. But,
the PDDB isn't performant enough nor targeted toward large
scale server applications. It should, however, be adequate for
pretty much any user data on a mobile phone if it were recompiled
for a 64-bit target. And, if you needed to scale up to bigger
files and/or more write cycles, the problem can be solved
with some simple memory allocation tricks on the large pool,
but because Precursor only has 16MiB of its own RAM to play
with, we don't implement that as the RAM overhead is both
detrimental and unnecessary at our scale of operation.
2021-11-25 17:56:39 +08:00
bunnie
ca485d5541 fix off-by-one in small pool to dictionary storage mappings
consolidate index math into helper functions to reduce errors.
2021-11-25 03:14:29 +08:00
bunnie
9d83b37818 unit tests for the free key index tracker 2021-11-24 14:50:44 +08:00
bunnie
8de0d70f43 debugging the patch-extend test.
probably should make a bunch of unit tests for the free dictionary
key tracking logic, it's too tricky to test out.
2021-11-24 03:57:50 +08:00
bunnie
800fcbe3bf fix page table consistency issue -- needs more testing to see if we licked it 2021-11-24 01:50:15 +08:00
bunnie
66a5b97705 revise pagetable entry to be 7 bytes wide -- this will allow
us to have a full 64-bit address space upon expansion of the page
number (turns out this is probably important because of the crappy
increment-only allocator used for the large blocks)
2021-11-24 01:49:22 +08:00
bunnie
42e926fcab fixes for patch update test
lots of corner cases found and ifxed
2021-11-23 23:33:06 +08:00
bunnie
472d60ae95 update a comment in the readme. 2021-11-23 15:55:27 +08:00