46ca14e2f9
commit |
||
|---|---|---|
| .. | ||
| locales | ||
| src | ||
| build.rs | ||
| Cargo.toml | ||
| README.md | ||
Graphical Abstraction Manager (GAM)
The GAM provides abstract UI primitives to other modules.
The goal is to have this module work in close conjunction with the
graphics-server, and all other modules would route abstract UI
requests through this module.
Structure
At a high level, you can think of the GAM as a firewall around the graphics-server.
The graphics-server has no concept of what pixel belongs where; it's happy
to mutate a pixel that is anywhere within the physical hardware framebuffer.
Giving processes direct access to graphics-server means that a less trusted
program could draw into an OS-reserved area, thus presenting false information
to a user. The GAM solves this problem by dividing the screen into Canvas objects.
Ux Context Names
See tokens.rs for a list of expected boot UX contexts. The GAM enforces a
policy that disallows the registration of unexpected UX contexts (including menus).
When adding more UX elements, be sure to expand the list of EXPECTED_BOOT_CONTEXTS,
or else the registration will fail.
Canvas
A Canvas is a minimal data structure that defines a physical region of the
screen that will display a set of primitives. Canvas structures are domiciled
in the GAM server, and are considered trusted by default, although there is
a flag that can be cleared to make everything within it untrusted.
Each Canvas has a 128-bit GUID. Application processes that wish to draw
something to the screen must refer to a Canvas by its 128-bit GUID; it is up
to the GAM to not share secure GUIDs with insecure processes. Thus the security
of a Canvas rests in the difficulty of guessing the 128-bit GUID, and also
in the system not leaking GUIDs.
Every GAM drawing object includes the GUID of the Canvas to which it should be drawn.
Upon receiving a draw request, it validates that the GUID exists, and applies
any other relevant rules (for example, a higher security process can use the
GAM to prohibit all drawing of lower security processes by marking their
Canvas as not drawable).
All GAM drawing objects specify pixel offsets from a (0,0) top-left coordinate
system. The GAM then handles translating these offsets from a virtual (0,0)
Canvas offset to a physical region of a screen through the clip_rect
record within the Canvas. The coordinate space of the clip_rect is fixed
to the screen's coordinates, that is, (0,0) in the top left, X increasing
to the right, Y increasing down.
A Canvas also stores a pan_offset. The pan_offset is added to every
coordinate inside the objects that refer to a Canvas and then the result
is clipped with clip_rect; this allows for easy implementation
of panning and scrolling. (Note: this feature is largely untested as of March 2021)
A Canvas has a trust_level associated with it. Higher numbers are more
trusted; 255 is the highest level of trust. Rules for drawing are as follows:
- More trusted
Canvasobjects always render on top of lower trusted object - When a higher trusted
Canvasobject overlaps a lower trusted object, the lower trusted object is:- defaced using hatched lines with a random angle and spacing
- further updates to the lower trusted object are disallowed.
Thus, a Canvas should not be thought of like a "window", as windows in
typical UIs are allowed to freely overlap and clipping is handled
by simply drawing over lower layers of content.
Canvas makes it strongly preferred to render trusted and untrusted data
side-by-side, rather than one on top of each other.
This policy is partially to help users be very clear as to e.g. where a password box is vs. an image that looks a lot like a password box; but the policy is also informed by the limitations of the underlying hardware. In particular, the underlying memory LCD strongly relies on "dirty bits" for good performance, and doing full-region redraws to handle dirty rectangles on window movement is not an efficient use of dirty bits. Reducing time spent redrawing partially obscured windows is also good for performance and helps to simplify the code base, but these last two considerations are quite minor compared to the primary concern of a "least confusing" UI when it comes to differentiating between trustable and less trustable regions of the screen.
Thus, the simple rule is: don't stack content types of different trust levels.
If you require content stacking, this can be done for content within a single
trust level by using multiple objects within a Canvas, as they have a draw_order
attribute and can handle content stacking; but between trust domains, it's both
a trust and complexity issue to allow for simultaneous stacking of trust domains
with live, full-content update of the underlying layers.
Defacing
Defacing is a policy implemented by the GAM which ensures that when trusted objects are overlaid on top of less trusted objects, the less trusted objects cannot be mistaken for a trusted object. This is to assist users with avoiding phishing attacks through Ux elements presented on less trusted canvases that look like trusted objects.
Defacing basically takes a bit of entropy from the TRNG and renders it in the form of random lines drawn across the defaced area. This allows users to still read the text within the defaced area, but makes it more difficult to mistake this area for something trusted.
Trust Level Policy Implementation
The RegisterUx opcode in gam is responsible for parceling out trust levels.
At boot, there is only a "boot context" -- no user apps are allowed, and all
contexts are assigned a high trust level. During the boot process, apps
are registered and claim names out of the tokens.rs EXPECTED_BOOT_CONTEXTS name space.
Roadmap (as of Xous 0.8.x): The idea is that once boot is thought to be completed, both xous-names and the gam
are checked to ensure that all the available slots for boot processes are occupied.
If this check passes, a flag should be set indicating we're
done, and the max trust level assignable should drop. This process disallows
future no-boot processes from registering as a trusted context, both in terms of their
name space and their trust level.
TextView
A TextView object is a heavy data structure that contains both a xous::String
and metadata which guides the GAM on how to render the string. Please note
the philosophy of the GAM is to hide the exact details of how anything is
rendered to the calling application. This both allows users to have greater
control over customizing their interfaces, and also helps introduce a layer
of protection against phishing; however it also means that UX designers will not
be able to have exquisite control over the "look and feel" of their applications.
TextView objects are domiciled on the application process. The application
process is responsible for guiding the rough layout of where TextViews go
in a canvas. Once the object is finalized, the TextView objects
are then mutably lent to the GAM using an rkyv lend wrapper;
the calling thread then blocks until the GAM completes the rendering operation.
For layouts that need to adjust in height based on variable-length text strings,
the calling application can use the bounds_hint/TextBounds to help manage this.
The bounds of a TextView can either be a fixed-sized rectangle, or a box that
grows up and out from a point plus a width. So, for example, a TextView could have
an anchor in the lower-right hand corner, plus a maximum width, and the height of the
box will be computed based as the text is rendered. The height of text can't be
known a-priori, because for example, emoji glyphs and hanzi characters will
have a different height than latin characters. A dry_run option is also
available for a TextView so one can simulate the rendering to determine the height
without paying the compuational price.
One can think of a TextView as a text bubble, that can have rounded or square
corners, and its content string can be rendered with a selection of options
that specify the glyph style (not a font -- it's more of a hint than a specifier),
aligment, and the size of the text bubble. The text bubble can either be of a
fixed size (such that the string will show ellipses ... if it overruns the
bubble), or of a dynamically growable size based on its content.
Thus, a typical "chat-style" app where text bubbles show a history of the chat going from the most recent at the bottom of the screen to the oldest at the top, would start by rendering variable-height text bubbles on the bottom, getting the returned value of the rendered height, and setting the height of the next bubble on top for rendering, and then rendering that.
TextView can both be directly rendered to a Canvas, or managed by secondary
object such as a Menu or List to compose other UI elements.
Menu
A Menu object encodes the state of a graphical menu. It's meant to be paired
with a dedicated thread that serves as an event loop. You can look at the
main menu implementation inside the GAM for an example of how this is done.
Most event loops will look about the same; if you do not want to build an
event loop, you can use the spawn_helper API to create one for you (see below).
Building a menu is fairly straight forward:
- register your
Menuobject's name insideservices/gam/src/tokens.rs. Currently, one can only register servers that are "expected" by the system; this is meant to prevent e.g. malicion code injection from conjuring UX elements. - create the
Menuobject by callingMenu::new(name: &str). This will automatically register an emptyMenulayout as a Ux element inside the GAM. - add menu items by calling
.add_item()on yourMenuobject - run the event loop and pass events back to the
Menuobject for processing
Each MenuItem has the following fields:
name: the text shown for the itemaction_conn: a xous CID to the target server that receives the action of the menu itemaction_opcode: the local opcode of the target server. Opcodes are private to the server, so menu-actionable opcodes need to be revealed with a helper method on the target server.action_payload: an enum that encodes the payload of the Opcode. For now, only Scalar payloads are implemented, but the framework is there for some kind of a Frankenstein static-buffer to be passed on to targetsclose_on_select: a boolean which indicates if the menu should automatically close once the item is selected
Modal
A Modal object encodes the state of a modal dialog box. A modal dialog box is limited to
this general format:
- Top text: describe the request
- Action: the interactive element
- Bottom text: reserved for validator error messages
The Action currently can be one of the following types:
- TextEntry: for passwords, or for regular text
- RadioButtons: for selecting one of many options
- CheckBox: for selecting any of many options
- Slider [NOT YET CODED]: for selecting a single numeric value along a range of values
Creating a Modal follows the same general pattern as the Menu, with the exception that the new() function is meant to be "complete": instead of creating a skeleton of a menu, the new() function takes all the necessary arguments for the repsective top, bottom, and action fields and tries to build the modal all in one go. It is, however, possible to dynamically modify the modal once created, using the modify() and remove() methods.
TextEntry supports an input validator; the validator takes as input the
proposed payload string, and returns None if valid, or an error message
that is plcaed into the dialog box's "bottom text" if the input is invalid.
TextEntry also supports "password" mode. When selected, text visibility can be controlled by a three-selection horizontal radio control that can select between "fully visible", "partially visible", and "fully obscured" states.
TextEntry is specifically coded so that its payload is cleared upon send, so that plaintext passwords are not left hanging around in the heap or stack.
The modals server allows applications to pop up generic modals without having
to write all the eventing glue code to the GAM. The server has the following properties:
- "Does about the right thing" for 90% of the applications
- Shared between multiple processes with a lock -- so the messages from this server should not be absolutely trusted
- Currently implements notifications, progress bars, text input, radio buttons, and checkboxes.
- Has a "pure Rust" blocking API so routines can sequence through the dialog boxes in a declarative fashion without having to write fancy sequencing logic.
Example code:
- The
rtcserver contains an example of a modal that uses themodalsserver. - The
root-keysserver contains an example of a static password modal, and uses an explicitly coded helper thread. It manages its own modals because it has a unique rendering property that can only be summoned by this process.
Helper Threads
A Menu or Modal needs to be able to respond to callbacks from the GAM. You can either do
this with a separate, explicitly coded menu loop (as seen is the main menu and the emoji menus),
or you can use the spawn_helper convenience call. To use spawn_helper, however, your
local server need to implement three opcodes:
Redraw- calls the.redraw()method on the modalRawkeys- handles a set of key events destined for the modalQuit- indicates that the server was told to quit for some reason (currently there's no good reason for this, but in the future once process destruction is supported, this will be important)
Your local server's opcode IDs are arguments to spawn_helper, along with both the private
and the public SIDs. The public SID is inside the Menu or Modal object; it's shared with the GAM. The
private SID is the SID of your loop handler, which is normally masked by xous-names. The private
SID should not be shared with untrusted processes. However, since spawn_helper is in your own
memory space, it's OK to shaer the private SID with this thread.
The TL;DR is that the helper thread is just a lookup table that maps UX opcodes to your thread's private opcode space, and it igonres any uknown opcodes.