diff --git a/tools/README.md b/tools/README.md index 7d76f0f98..d9292d5e3 100644 --- a/tools/README.md +++ b/tools/README.md @@ -80,9 +80,11 @@ For more about `i18n_helper.py` please see the locales [README](../locales/READM ## Testing -### Running `pddbci.py` (PDDB Regression Tests) +The director is overloaded with some Python scripts used for PDDB tests. These probably should be moved to a more appropriate location in the future. However, it is what it is for now. -`pddbci.py` drives the PDDB regression suite. It calls `cargo xtask pddb-ci` +### Running `pddbci.py` (PDDB Regression/Fuzzing Tests) + +`pddbci.py` drives the PDDB regression & fuzzing suite. It calls `cargo xtask pddb-ci` to generate test images and then runs `pddbdbg.py` to verify them. The analysis step requires Python 3 and several third-party packages. @@ -106,23 +108,30 @@ missing build files): ```sh pip install pycryptodome cryptography six -pip install git+https://github.com/painor/pyaesni.git ``` +Note: `pyaesni` was deprecated because the package does not install on all operating systems. The AES routine has been reverted to the standard Python AES routine which causes the test loop to run a bit slower. + #### 4. Run the regression tests Choose the `--name` of a database image from `tools/pddb-images` +The database images are generated by code that runs here: + +https://github.com/betrusted-io/xous-core/blob/c105531e48c59159c4051fdd9c5e539a5c36377e/services/pddb/src/tests.rs#L592-L596 + +The images are incremental snapshots of various tests working through the PDDB. In practice, the final test (`basis2`) is the test one would run the fuzzing/regression suite on, and in case of a regression, then one would analyze earlier snapshots to work out the root cause of the regression. + ```sh # From the repository root, with the venv active -python3 tools/pddbci.py --name full +python3 tools/pddbci.py --name basis2 ``` The `--runs` flag controls iteration count (default 501). Use a smaller value for a quick smoke test: ```sh -python3 tools/pddbci.py --name full --runs 5 +python3 tools/pddbci.py --name basis2 --runs 5 ``` You can adjust verbosity with `--loglevel DEBUG`.