Instead of getting the slice by creating a large Vec, I now use
MapMemory to create this region directly. There is some sort of bug
with Vec that doesn't allow creation of large Vec's, and this change
makes it possible to load much larger apps (including `vault',
although there are still issues like menus to sort out).
I have added another progressbar when the list of apps is being
loaded (I find that this makes stuff more clear).
I also discovered that if I use gam to "switch" to app-loader from
itself this works at bringing the menu back up
`app-loader' can now successfully download apps from the app server
and run them! I have also improved the UI by adding a progressbar
when an app is loading so that you know when it is done and also
replacing some panics with notifications that way the whole machine
doesn't shut down if the server has some problems.
The remaining UI work is to make sure that the menu pops back up when
you for example set the server.
I also need to add locales, a README, etc.
This creates:
1) An `xtask' command that allows someone to compile individual apps
without compiling the entire image (note that this is required because
xtask generates files that contain gam context names, and if you just
do a regular `cargo build' building the app will fail)
2) A python file app_server.py that acts as a server to give the
app-loader a list of the apps and actually send it the
3) A basic UX that allows app-loader to set the server, get the list
of apps, and load apps.
Currently, this mostly works, except that there is some random
`ShareViolation' when the app-loader loads the app.
I have made `app-loader' more object oriented (to fit in with the
style of the other xous apps), and I am moving some of my personal
projects to another branch of my repo
`EXPECTED_APP_CONTEXTS' was added to the list of authorized
switchers. I don't think this is ideal, but I think it would be
difficult to come up with a better solution that doesn't make
`app-loader' "special" in some way.
This allows `app-loader' to register new names (which are then used by
the loaded apps to claim tokens). At the moment this is restricted to
apps that are pre-registered, i.e. included in the image.
It turns out that in ELF files there *is* a difference between
file size and mem size. Mem size is the size of the chunk of memory,
but file size is the size of what we copy into there from the
file. Most of the time they are the same, which is why previously
`spawn' only considered mem size, but this caused issues.
In particular, in the xous implementation of the rust standard
library, there whenever something is allocated it has to first check
an AtomicI32 LOCKED to see if it is 0 or 1, and set it to 1 when it is
allocating so that only one thing is allocating at a time. However,
the mem size for the section containing LOCKED is larger than the file
size, meaning that `spawn' copied more of the file that it should
have. This lead to junk being put into LOCKED, which caused the new
process to assume that something else was perpetually allocating.
Now, `spawn' puts 0's in the space between the end of the file size
and the end of the memsize, which makes everything work as expected.
The only problem remaining is that when an app registers for GAM, GAM
checks whether the app is allowed to register, which is based on what
apps were included in the image. Obviously, this does not work when
you are trying to load apps that are not already in the image
I forgot to revert to the previous versions of the other status
files (turns out I only need to do it for one). I also changed to
using the hello binary in debug.
When the user selects App Loader from the app menu, a new menu shows
up that displays the currently loaded apps (in the future this will
contain options to add/remove apps as well). This is a much more sane
approach than whatever I was doing before because it requires minimal
changes to the rest of the source tree (I had to change xtask so that
it would replace -'s with _'s in constant names so that I could name
my app app-loader).
when the font menu is raised while in TOTP mode, the "pump"
thread continues to prompt for redraws, which can eventually lead to
an overflow of the server queue on the main thread that leads to
deadlock and eventually a crash.
This patch does two things:
- splits the allow_host variable and the allow_totp_rendering variable
so that the two flags have distinct meanings
- use the allow_totp_rendering variable to control when the
totp thread is allowed to pump the main thread to animate the
timer bar.
Previously the allow_host variable was introduced to control access
to sensitive data, and it incidentally was also used to control
TOTP rendering because when host readout is activated this can
freeze the UI for extended periods of time.
By splitting the flag we can now control both independently of
each other. Rendering still stops when host readout is enabled,
but it also stops when any other blocking dialog boxes are popped up,
hopefully preventing future crashes if a user is distracted in
the middle of a UX flow that can block the main loop.
default system font can be statically changed by modifying
the SYSTEM_STYLE constant.
This of course will lead to a request for a dynamically
configurable preference for font size. However, early
on it was decided this is a bad idea. Here's why.
We don't provide for dynamic system-wide font selection
because this adds a lot of complexity to what would otherwise
be static code: this is a deliberate choice to improve code
density and performance. If the system font were dynamically
adjustable, then every character rendered we would potentially
have to query the PDDB for the current system font setting.
The impact of this could be mitigated by caching values but
then what happens when you actually change the font? now
you need a registry to fire messages off to all dependent
crates with cached values to flush their caches and redraw
(or you punt and force a reboot if you change font sizes,
but still, you're creating a dependency on a lot of plumbing
for even the simplest crates).
Plus, with a static font selection, the compiler can take constants
like the font height and compose them with other fitting
parameters into optimized load/stores of borders and spacings
for the static case; if you make it dynamic, all those multiplies,
adds and divides now get pulled into the main rendering loop
and are recomputed all the time. For example, if you have a static
const list of items that need to be rendered, even if the code to
compute the height of the list is written as a for loop, the compiler
will know the list is constant, and the font height is constant,
and it can condense that all into a single number at compile time,
eliminating all the code for the for loop.
If you leave the option open to change that, that entire piece
of code has to be generated and executed even though in practice
nobody changes the parameter.
This change is made with the intention of making the xous-rs crate a bit
more freestanding without depending on the xtask build system.
Signed-off-by: Jean-Pierre De Jesus DIAZ <me@jeandudey.tech>
rely more on the cache retaining consistency with the disk
versus always reloading from the disk to speed up performance.
might be useful to make a routine that can also check coherency
periodically, because with the new format the cache could
co-exist along side the database for a very long time without
any checking.
also eliminates references, copying and locking of items in the
sorted btreemap.
turns out that ... sorting is much cheaper than copying or maintaining
references, because copies to heap items require allocs which are
under the hoood several sorts to find the place to stick the alloc
memory, and shared references require locks.
- make the "base" field no longer public. It is accessible, but only
through an `unsafe` method, because, it is unsafe.
- add accessors to offest/mask fields. Because, it's not at all
unsafe to access these, and it can be useful for bit-ops that are
not part of the generic set of UTRA.
- add Arc to AtomicCSR, and require `std` for AtomicCSR types.
This should not break anything, I checked inside @foundation-devices
keyOS and they don't seem to be using AtomicCSR anywhere.
The Arc makes it a bit safer to clone the AtomicCSR base pointer
and share it between threads, but it will have an impact on performance.
The USB stack (which requires AtomicCSR) was regression testing and the
impact seems minimal.
Recall that the main purpose of AtomicCSR is to provide a CSR accessor
that does not require a mutable reference to the base object. This
can make certain Rust idioms a bit cleaner. We wrap the base accessor
in Arc+Atomic to try and improve the safety of throwing the borrow
checker under the bus, but it's still not perfect because hardware
mutable state can have side-effects within hardware itself that cannot
be modeled or checked by Rust's borrow checker.
TextEntry boxes can now specify "growable" as an option; if this
is set true, then, the fields within are considered to be multi-line
fields that can grow to fill the screen (even pushing lower ui
elements off the screen...); however a text line is limited to
256 characters so generally the overflow issue is hopefully rare-ish.
Left/right arrow keys conjure up an insertion point to allow for
deleting and inserting text in the middle of a field.
The main quirk is that if you enter text that causes a field to grow in
size, the overall dialog box can't grow any larger, so the text simply
overflows into the UI element above the current line, without limit.
Need to check for interactions with other elements that should
not be affected but it passes some simple test cases in `vault`.
If the "notes" field starts with the characters
"bip39" (case insensitive), then, the password field is
interpreted as a hexadecimal encoding of BIP39 entropy.
To create a BIP-39 entry, leave the password field blank,
and then start the Notes field with the characters 'bip39',
and this will trigger a dialog box that will only allow
valid bip39 entry and check the checksum etc.
To view the BIP-39, after opening up the entry for "edit",
close it, and it will pop up a second dialog box that interprets
the hex data as BIP-39 words.
You could also in theory enter the hex digits yourself into
the password field, if you're masochist.
The crate just re-exports the com_rs crate. This patch changes usages of
com_rs-ref to use com_rs directly.
Signed-off-by: Jean-Pierre De Jesus DIAZ <me@jeandudey.tech>
0.5.1 rand_core needed for the dalek crates
0.6.3 traits everywhere else
what a mess
turns out i can just use fill_via_next() to do the same thing
without having to step on cross-generation traits. whew.
(also not in tree but doing a quick manual test that fill_via_next()
is not totally broken)
this adds a "kicker" thread that gets updated with a target time
to kick the UX by the FIDO2 USB handler. When the target time is
met, an "update" command is issued to the UX.
This hopefully creates an efficient mechanism for triggering a
UX redraw that also has an acceptable delay.
This may not be the correct fix, but it causes the test bench to pass.
This will be revisted once I hear back from the OpenSK maintainers.
I strongly supect the issue is actually a bug in how permissions are cleared
after a user presence test, and not that the checks themselves are
incorrect.
currently 10 seconds from boot before cred reset can be issued;
automated testing can't always keep up with that. expand to
20 seconds. This is not "standards compliant" but the idea that
you unplug the device and it stays on is also not "standards compliant".
Code now builds, but, it doesn't fully work. Also,
the vault backup vendor routines have yet to be wedged in.
On the other hand, this edit is much more targeted than
the first one, so hopefully future port-ins of OpenSK
will be much smoother. In particular, OpenSK has a nice
`env` abstraction that has enough power now for us to
stick most of the Xous-specific stuff in there. There's
still a few patches needed to the main code loop, mainly
to accommodate our ability to display more information
than flash an LED.
The storage layer is also migrated to a totally new
dictionary, because the latest version also completely
redoes the storage with some sort of key/value system,
which maps nicely into the PDDB.
Added more predefined scalar messages
Checks WiFi status
Spawns separate thread for client_sync (reading the network)
Most all log messages are now INFO (to be shown by default)
Closes#300
Signed-off-by: Tom Marble <tmarble@info9.net>
Allows a variable to be set to the EMPTY string
- by not setting a value `/set myvar`
During initialization now verifies that PDDB is mounted.
Added warnings for PDDB not mounted and WiFi not connected.
- the WiFi check is not yet implemented
Signed-off-by: Tom Marble <tmarble@info9.net>
Adds migrations framework
Delays mtxcli initialization until first user input, because we know:
- PDDB is mounted
- Timer is stable
- User actually using mtxcli
Adds send_async_msg scalar_async_msg
- basis for automatically updating chatroom
Signed-off-by: Tom Marble <tmarble@info9.net>
Adds migrations framework
- Migration for v0.9.11-0120: update the filter
Filter
- narrow filter to only the fields and events for the current room
Introduces "read only" variables (e.g. __version).
Signed-off-by: Tom Marble <tmarble@info9.net>
`vault` is changed so that instead of clearing the list of
data each time the screen is updated, it tries to compare it
against the existing data and modifies the record if it needs to,
or leaves it untouched.
Furthermore, at least in the password path, temporary strings
are pre-allocated and re-used, instead of freed and re-allocated
every time through the main loop.
The combination of these optimzations reduces the time to re-parse
a couple hundred records from about 1000ms to 200ms, reducing the
overall time to retrieve the database from 1.8s to 1.0s. It also
makes the loop much more stable over time; previously, as the heap
fragmented, performance would slowly degrade. Now that most
records are static and re-used, the performance seems to stay
constant.
The framwork is also put in place to re-use TOTP/U2F records,
but the code still uses the legacy method of clearing the
entire stored list and rebuilding it. However, it wasn't deemed
a priority to fix this because generally I haven't seen use cases
where users have hundreds of tokens or TOTP's; so long as the
number is about 20 or so, the performance difference should
be imperceptible with or without optimization.
This refactors the remaining vault accessor routines to use
the new more efficient key pre-fetching routine. This also
fixes up an error in test case generation and hopefully resolves
an issue with concurrency errors using the older version of
the access routine.
* Updates the mtxcli app
mtxcli:
- Simple chat now functional!
- All strings use translations
- Updates the README
- updates mtxcli to handle PDDB existence check
Adds *.pyc to .gitignore
Signed-off-by: Tom Marble <tmarble@info9.net>
results are slightly better on the first run (3700ms instead of 4900ms)
but there seems to be really excessive allocations using the heap
allocated serializer by rkyv. We really need to use the bufserializer
I think for this to work.
In general, hosted mode is assumed "by default" if the target
is not Xous. This allows packages to be built cleanly on command
lines that don't know to pass a --feature flag to specify a target.
However, an explicit "hosted" flag is still retained so that we
can confirm the build system intent. This saves us from the failure
mode that we thought we were building for target A, but accidentally
ommitted the flag, and instead we've been silently building for
"hosted" mode without any warning. The main consumer of the "hosted"
flag is now just the build.rs script inside of the utralib.
* Adds mtxcli commands using the PDDB
/get key -- shows value of key (from the PDDB)
/set key value -- sets value of key (in the PDDB)
/unset key -- removes key (from the PDDB)
/status -- shows status of the connection
Updates .gitignore for PDDB in hosted mode
NOTE: action on DefaultApp is not yet implemented: this
should allow the GAM to choose which application is safe
as a default (for other applications to "quit" by switching to it).
Proposed default application is "shellchat".
Updates mtxcli commands
- removes audio command
- adds help and quit commands
Initial version of mtxcli is simply a copy of repl with the
requisite manifest changes to make the app unique.
Signed-off-by: Tom Marble <tmarble@info9.net>
Avoid allocations in some cases, for example, when formatting
to an I/O buffer. Or when formatting an string.
Signed-off-by: Jean-Pierre De Jesus DIAZ <me@jeandudey.tech>
This will allow for e.g. xous on riscv32-imac but not a precursor
target, for example: by using feature selectors we get to have
a bigger range of targets that we support, such as for implementations
that haven't been upstreamed or have official target-triples.
The downsides are:
- patch override libraries don't absorb feature flags. so, for
example, the AES crate that is used to patch other crates won't
be aware of the target features without modifying the referencing
crates.
- features are normally meant to be additive, but target features
are mutually exclusive. The mutex is enforced by a check in
utralib that throws a panic on build if two targets are accidentally
selected at the same time.
- HOTP now prompts for an initial count
- Found a code path where improperly formatted HOTP field can
lead to a 0-value being passed to a totp generator
- Added guardrails around the TOTP generator so it can't divide by 0:
it issues a warning instead.
hotp is merged as a "minor" feature off of totp. In other words,
the default mode is totp, but one can specify a totp record
to be an hotp record and it will behave as an hotp record.
We re-use the "step_seconds" field to specify the count parameter.
There is no separate initial count / count field -- the two are
tracked in a single u64 that is stored in the "step_seconds" field
of what would be a totp record.
This also revs the TOTP record to v2. This *should* not break
anything:
- v1 records deserialize as totp records into v2 (which is correct,
as there are no hotp in v1)
- when a v1 record is saved to disk, it is stored in v2 format
- v1 records that are not edited stay as v1
unique content
Instead of generating a unique GUID every time a new record gets
created, hash known unique content for every entry and use that as key.
The StorageContent trait now has a hash() method that returns a Vec
containing the SHA-256 hash of the object's unique content.
Hashing is done by re-using the same calls that ctap uses.
This allows dupes detection when restoring a backup, allowing us to
employ a "never overwrite" policy: Precursor is the main source of truth
for such data.
Still needs a way to deterministically generate a key for each entry.
Idea: a `hash() -> String` method on StorageContent so that new_record
can call it, store said object with the newly-generated key.
Since this is a deterministic operation, restoring a backup with dupes
would not overwrite what's present in PDDB already.