xous-core/tools/encrypt-bitstream.py
Gianguido Sorà 46ca14e2f9
Squashed commit of the following:
commit a09100ab4a
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:55:55 2022 +0800

    bump lockfile

commit 53172e1471
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:46:40 2022 +0800

    bump verification pins

commit 36398810dd
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:24:36 2022 +0800

    bump version numbers to absorb new syscalls

commit 062fa6178a
Merge: ac4a854e 3bcc955a
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:23:31 2022 +0800

    Merge branch 'reply-and-receive-next-syscall' into main

commit ac4a854e49
Merge: d1375343 e4c298db
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:22:25 2022 +0800

    Merge branch 'main' of github.com:betrusted-io/xous-core into main

commit 3bcc955aba
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:21:43 2022 +0800

    add test code that also benchmarks the new routines.

commit e4c298db40
Merge: f71278b4 e2bf7109
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 30 15:16:13 2022 +0800

    Merge pull request #289 from betrusted-io/reply-and-receive-next-syscall

    Reply and receive next syscall

commit e2bf710939
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:06:51 2022 +0800

    services: xous-ticktimer: use reply_and_receive_next()

    Convert the ticktimer server to use the new
    reply_and_receive_next_legacy() call. This is an example of how to use
    this API for future projects.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 0e75b8e479
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:05:26 2022 +0800

    test-spawn: rename `Message` to `MessageEnvelope`

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 3218bde2b4
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:04:01 2022 +0800

    kernel: implement reply_and_receive_next

    Implement `ReplyAndReceiveNext` as a syscall. This will reply to a
    message, pull the next message out of the queue, and then re-queue the
    server to run. This should make servers much more responsive.

    This patch has been tested on both hosted and native modes.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit d1375343c1
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 23:34:48 2022 +0800

    update release notes

commit f71278b422
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 22:04:55 2022 +0800

    fix issue #109 - refactor key fill/scan to not crash after pruning

    After a prune, key fill/scan can crash because there is an assumption
    that data is always resident in the cache for a small key.

    This code handles the case where the small key data was evicted during
    a prune, and reads the data back in just for that key on demand.

    Furthermore, during a `sync` call, it will pull in the key data
    that corresponds to the page containing just the small key, allowing
    that page to be re-composed and replaced in memory. caching is used
    on the decryption of the small key page so the expensive bit
    of the call should only happen once per small key bank.

    tested with pddb-ci and manually in hosted mode.

    A portion of code was re-written to make it clear that only the
    length of allocated data is copied to/from disk and memory, instead
    of the reserved space as the previous code would indicate. The reserved
    space is guaranteed to be zero because the RAM-based page for composing
    the small key block is zero-ized prior to operations; left comments
    reminding me of this in the future so I don't accidentally optimize
    something out and cause data leakage between small keys because I'm
    allowing the reserved areas to contain stale data.

commit e25da3a720
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 22:04:15 2022 +0800

    add implemenation of the `prune` debug call

    this allows us to trigger the cache prune manually, instead of
    in response to memory backpressure only

commit 549e67fead
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 22:03:19 2022 +0800

    add a `prune` debug call to force a hard-prune of the pddb cache

commit e045574d39
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:06:51 2022 +0800

    services: xous-ticktimer: use reply_and_receive_next()

    Convert the ticktimer server to use the new
    reply_and_receive_next_legacy() call. This is an example of how to use
    this API for future projects.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 23c17d7632
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:05:26 2022 +0800

    test-spawn: rename `Message` to `MessageEnvelope`

    Signed-off-by: Sean Cross <sean@xobs.io>

commit aae5f3737c
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:04:01 2022 +0800

    kernel: implement reply_and_receive_next

    Implement `ReplyAndReceiveNext` as a syscall. This will reply to a
    message, pull the next message out of the queue, and then re-queue the
    server to run. This should make servers much more responsive.

    This patch has been tested on both hosted and native modes.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit ab0c7d3b32
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:03:39 2022 +0800

    kernel: server: check buf is some when returning mem

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 46f340324d
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:02:57 2022 +0800

    kernel: rename `Message` -> `MessageEnvelope`

    This syncs with the latest changes to `xous-rs`.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 911ddb60fc
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:01:27 2022 +0800

    kernel: debug: fix klog on baremetal

    klog() refers to an outdated `.uart` struct, which has since been
    renamed to `.serial`.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit d88254b1b1
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:00:35 2022 +0800

    xous: add reply_and_receive_next_legacy call

    This call is used to return values while simultaneously listening for
    new messages.

    The -legacy suffix is used to disambiguate scalar values, so we know
    whether to send Scalar1, Scalar2, or Scalar5 messages.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 64c88f9942
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 21:00:05 2022 +0800

    xous: add Message type

    This type actually contains the message contents, and will be used to
    return data to the caller.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit d463f9563f
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 29 20:58:51 2022 +0800

    xous: rename Message to MessageEnvelope

    This reflects the fact that up until now, the `Message` struct has
    merely been the envelope that surrounds the message. It does not contain
    any actual Message contents.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 49ce85f9a8
Author: Sean Cross <sean@xobs.io>
Date:   Wed Jul 6 15:57:10 2022 +0800

    ticktimer-server: prepare for using ReplyAndReceiveNext

    Misc cleanup prior to switching which call we use.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit be56af6303
Author: Sean Cross <sean@xobs.io>
Date:   Wed Jul 6 15:35:22 2022 +0800

    xous: add wrapper for reply_and_receive_next

    This wrapper enables userspace programs to use the new
    ReplyAndReceiveNext syscall easily.

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 36e6e6c30b
Author: Sean Cross <sean@xobs.io>
Date:   Wed Jul 6 15:18:32 2022 +0800

    xous: syscall: add ReplyAndReceiveNext call

    Signed-off-by: Sean Cross <sean@xobs.io>

commit 1e56598b10
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 00:53:32 2022 +0800

    chore: fix some typos

commit 1291b0f00b
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 00:38:27 2022 +0800

    add issue # to release note

commit 7206f24271
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 00:37:47 2022 +0800

    update release notes

commit 2f30e1b50a
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 00:11:03 2022 +0800

    implement #269

commit 5370c333da
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 29 00:10:42 2022 +0800

    add callback for monitoring basis lock/unlock events

commit 8d42feb051
Author: bunnie <bunnie@kosagi.com>
Date:   Wed Dec 28 19:48:40 2022 +0800

    increase heap for mtxcli; add memory usage reporting

commit ac1df054ab
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 26 13:52:44 2022 +0800

    absorb 22-bit satp fix

commit 89500015a7
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 15 17:52:55 2022 +0800

    try to fix regression from #284

    still hitting cache coherence edge cases. expanding the flush
    set read by one more way.

commit 41ea0a37c5
Merge: e422740d b288d93a
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 15 14:36:40 2022 +0800

    Merge branch 'main' of github.com:betrusted-io/xous-core into main

commit b288d93abd
Merge: 8e1d0502 d4527b82
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 15 14:00:10 2022 +0800

    Merge pull request #284 from betrusted-io/use-thin-lto

    cargo: use thin-lto

commit d4527b8227
Author: Sean Cross <sean@xobs.io>
Date:   Thu Dec 15 12:26:00 2022 +0800

    cargo: use thin-lto

    Use thin LTO for linking. This shouldn't have too big of an affect on
    the resulting binary size, and it fixes the build for 1.66.0:

    = note: rust-lld: error: undefined symbol:
    core::panicking::panic_str_nounwind::h9b449c0fe03c8dac
    >>> referenced by compiler_builtins.22a3dcb1-cgu.13
    >>> compiler_builtins-b913bacb8fda1c55.compiler_builtins.22a3dcb1-cgu.13.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h3db6431f23a281c1)
    in archive root\lib\rustlib\riscv32imac-unknown-xous-elf\lib\libcompiler_builtins-b913bacb8fda1c55.rlib
    >>> referenced by compiler_builtins.22a3dcb1-cgu.13
    >>> compiler_builtins-b913bacb8fda1c55.compiler_builtins.22a3dcb1-cgu.13.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h3db6431f23a281c1)
    in archive root\lib\rustlib\riscv32imac-unknown-xous-elf\lib\libcompiler_builtins-b913bacb8fda1c55.rlib

    Signed-off-by: Sean Cross <sean@xobs.io>

commit e422740d37
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 23:03:50 2022 +0800

    chore: last one. really.

commit ad79ebf8ba
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 23:02:09 2022 +0800

    chore: more typos. no really, i can stop at any time.

commit b1b6d46d27
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 22:56:37 2022 +0800

    chore: more typos

    probably should not have tried to spell check this, now i'm wasting
    my time.

commit 283ea1de0d
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 22:49:25 2022 +0800

    chore: fix more typos, add french dictionary to vscode workspace settings

commit 1d94968b2f
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 22:36:47 2022 +0800

    chore: fix typos

commit e930d88563
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 01:42:00 2022 +0800

    placeholder for 0.9.12 release notes

commit 8e1d05020d
Author: bunnie <bunnie@kosagi.com>
Date:   Tue Dec 13 00:32:11 2022 +0800

    add v0.9.11 to the restore.py URL list

commit bd230f5bb0
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:51:27 2022 +0800

    fix text to speech mode preferences setting

    never allow the audio system to be turned off for blind people,
    and disallow 0-volume setting as well.

commit 38bc254bb9
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:28:33 2022 +0800

    fix missing ja/zh translation

commit 60d4482ce2
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:16:04 2022 +0800

    bump lockfile

commit 56f0843229
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:08:28 2022 +0800

    bump verifier numbers

commit 5370ce396d
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:07:20 2022 +0800

    bump all revs due to SoC artifact update

commit cac0e57cda
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:06:31 2022 +0800

    change default target

commit 04b441a234
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:06:19 2022 +0800

    add 0.9.11 build artifact

commit 08cf863ff5
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 16:05:15 2022 +0800

    update SoC artifacts for 0.9.11 build

commit c32b5d5bac
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 15:31:15 2022 +0800

    bump release notes

commit 80ce119c95
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 15:31:06 2022 +0800

    fix RSSI calculations

commit edf24d9fc2
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 14:11:19 2022 +0800

    clean up on the wifi signal bars

commit eb17edc4ec
Merge: 9a580070 435a6ea2
Author: bunnie <bunnie@kosagi.com>
Date:   Mon Dec 12 11:47:18 2022 +0800

    Merge pull request #283 from gsora/feat/wifi-signal-bars

    feat: wifi signal bar

commit 9a580070a3
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 17:50:51 2022 +0800

    revert threaded init question now that deface issue is fixed

commit f9cdf31857
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 17:07:29 2022 +0800

    bump release notes

commit 196ceb0d29
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 16:42:03 2022 +0800

    fix: avoid thrashing when recomputing previously defaced canvases

    There was a dubious line that I never felt good about and today
    I found out why. By flipping every canvas to drawable before doing
    the intersection computation we also throw away the defacement state,
    so previously defaced canvases would lose their status bit, causing
    thrashing of the canvases anytime there is a re-computation of
    the canvas ordering.

    This fix caches if a canvas was already defaced, and if it turns out
    that an on-screen canvas should not be drawable but was already
    defaced, the setting is re-applied, preventing thrashing.

commit dca043d387
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 16:41:53 2022 +0800

    improve context swap debug messages

commit 8d8dd2bd2f
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 16:41:25 2022 +0800

    fix typo in unwrap message

commit f31afa5125
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 11 16:41:06 2022 +0800

    remove unused opcode in shellchat; improve debug output

commit 6138b65d76
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 23:44:18 2022 +0800

    update comments to reflect changes in CI infra

commit 399124f942
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 22:05:47 2022 +0800

    fix hosted mode

commit 939e13950d
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 18:59:55 2022 +0800

    comment out debug in rootkeys implementation section

commit 2097afc166
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 18:58:06 2022 +0800

    add efuse burning flow to rootkeys

    currently only available for builds that have the `efuse` feature
    enabled

commit 19e16aea29
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 18:56:59 2022 +0800

    fix jtag fuse burning & add diagnostic messages

    - msb/lsb swap on efuse bank fixed
    - hazardous-debug feauter gates added
    - dry-run for "testing only" added

commit 4ec866e1d7
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 18:56:12 2022 +0800

    add diagnostic seal command

commit f5c687a524
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 17:45:30 2022 +0800

    clarify user prompt

commit 2b4e47384e
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 17:44:54 2022 +0800

    add fused device support and warnings to precursorupdater

commit e42c4c7b33
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 16:39:14 2022 +0800

    add warnings & support for fused devices

    burning a generic image to a fused device will surely lead to
    a brick, so, try to keep a user from shooting themselves in the foot.

commit c2f7a63f97
Author: bunnie <bunnie@kosagi.com>
Date:   Sat Dec 10 16:39:01 2022 +0800

    remove some whitespace to allow for easier debugging

commit 1f89843122
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 9 12:38:53 2022 +0800

    defer init root keys question until status thread starts

    also defer the preference thread from loading until after the
    PDDB mount thread is started -- doesn't make sense to start it
    any earlier than that, since it depends on the PDDB being mounted.

commit 02f9701546
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 8 17:42:12 2022 +0800

    make an explicit dvt feature to simplify flags

commit 4dd983bfc4
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 8 17:19:05 2022 +0800

    adjust security parameters for dvt build

commit fc08c3e29d
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 8 17:14:47 2022 +0800

    add hazardous-debug to modals bip-39

    for convenient key printout during debug. flag is normally off.

commit dd82b7f32a
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 8 17:14:21 2022 +0800

    add no-codec mode to status

    this is for DVT builds where the codec isn't supported, needed
    for efuse testing

commit a5d341c7a6
Author: bunnie <bunnie@kosagi.com>
Date:   Thu Dec 8 14:09:36 2022 +0800

    add a bitstream encryption/recovery tool for fused devices

    This is useful for recovering bricked devices where the backup
    key is available. This tool can be used to generate an FPGA
    image for a device that has been permanently fused to a non-0
    backup key.

commit 4f270f54be
Author: bunnie <bunnie@kosagi.com>
Date:   Wed Dec 7 22:24:56 2022 +0800

    revert debugging on status opcodes

commit 435a6ea2bd
Author: Gianguido Sorà <me@gsora.xyz>
Date:   Wed Dec 7 14:28:07 2022 +0100

    feat: wifi signal bar

    This commit draws a Wi-Fi signal bar with 5 bars (lowest to highest signal), to the right of SSID name.

    Signal power is calculated from a lowest of -90dBm to a maximum of -30dBm, by increments of 13dBm.

commit f84a9d10c7
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 4 18:13:37 2022 +0800

    add placeholder for the efuse flow in the rootkey service

commit 8208a86c08
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 4 18:12:30 2022 +0800

    add the UX front-end for backup key burning

    - instructions for BBRAM flow (refers to wiki)
    - consent for efuse flow
    - fix a blocking issue on backup prep (minor problem, just gives
    users more time to contemplate without a panic)

commit 3838f3bc15
Author: bunnie <bunnie@kosagi.com>
Date:   Sun Dec 4 18:12:05 2022 +0800

    add a pause for context swaps on menu release

commit 53c9af1033
Author: bunnie <bunnie@kosagi.com>
Date:   Fri Dec 2 23:00:38 2022 +0800

    bump readme's
2023-01-02 15:51:45 +01:00

526 lines
No EOL
22 KiB
Python
Executable file

#!/usr/bin/python3
import argparse
from Crypto.Cipher import AES
from Crypto.Hash import SHA256
from Crypto.Random import get_random_bytes
import binascii
import logging
import sys
from bip_utils import (
Bip39MnemonicValidator, Bip39MnemonicDecoder
)
from pathlib import Path
"""
Encrypting a 7-series bitstream.
Encrypted bitstreams have a slightly different structure than
normal bitstreams, so to simplify things, we start from a bitstream
that is "encrypted" with a dummy key. We then decrypt this stream
and re-encrypt it to the key of choice; by doing this, we can just
update the file with minimal editing.
We working with a .bin format, which lacks the informative header
on the .bit file (and is the type that is burned into the ROM anyways).
# Decryption
The ciphertext starts at byte 184. To decrypt:
* divide into 32-bit chunks
* reverse the order of the bits in each 32-bit chunk
* feed into AES in big-endian order
So, to encrypt:
* feed bitstream into AES blocks in big-endian order
* reverse the order of bits in each 32-bit chunk
* write out in big endian format. This is referred to "as-stored" format.
This is validate with key=0, IV=0.
# The HMAC
The HMAC key is written into the bitstream as follows:
Two copies of the HMAC key are stored, once in the header, once in the footer.
Prepare the header:
* flip the order of bits in every 32-bit chunk of the HMAC key
* XOR each byte with 0x6C
* pad an additional 32 bytes of 0x6C
* pre-pend to bitstream
Total header length is 64 bytes.
Determine the last command in the bitstream:
This is probably the last instance of "00 00 00 04" in the as-stored bitstream.
You'll know you found it because when looking at a decrypt bitstream, immediately
after this you will see a SHA-256 padding (1 followed by many 0's then the
length of the message; note the order that comes out of AES has to be
bit-flipped for this pattern to be obvious, when staring at the as-stored bitstream
it's not totally obvious).
The padding looks something like this as-stored:
v the '1' bit per SHA requirement, but bit-flipped
0000 0001 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0026 d080
^ length of message in bits
Compute hash of the bitstream from the very first byte of the header
to the end of the active bitstream + SHA padding. Call this "hash1".
Append an additional 256 bytes of 0's after the end of the padding that was
required to compute hash1.
Now, prepare the footer:
* flip the order of bits in every 32-bit chunk of the HMAC key
* XOR each byte with 0x3A
* pad an additional 32 bytes of 0x3A
* append hash1
Now, compute a hash of the region spanning from the beginning of
the footer (not including the 0 pad, so starting at the 0x3A sequence)
to the end of hash1. This means first padding the region, and then
computing the hash.
Bit-flip and append this hash to the overall file, and you have now
an HMAC-ready bitstream. Note that the Xilinx implementation 0's out
the copy of "hash1" as stored in the bitstream.
Once prepared, this entire set is encrypted using AES-CBC.
Note that the number 0x085b98 is immediately before the cipherext data (this is for
the 35T), and this corresponds to 547,736 words or 2,190,944 bytes, which corresponds
exactly to a 32-byte SHA-256 digest appended to the end of the entire stream including
header and footer (including the extra weird 64 bytes at the end), that is, 2190912 bytes.
An extra 16-bytes of random-looking data appears after this length when you
"over-decrypt" because the repeating output of the CBC cipher doesn't start until
one block after the last block.
Bitstream footer notes:
00216d90: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00216da0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00216db0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ <-- end of FPGA bitstream
00216dc0: 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a :::::::::::::::: | | <-- hmac key xor with 0x3a
00216dd0: 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a :::::::::::::::: | |
00216de0: 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a :::::::::::::::: | |
00216df0: 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a 3a3a :::::::::::::::: | |
00216e00: 0000 0000 0000 0000 0000 0000 0000 0000 HMAC1........... | | <-- where hmac digest #1 was: SHA256(bitstream)
00216e10: 0000 0000 0000 0000 0000 0000 0000 0000 .......was here. | |____ 96 bytes = 0x300 bits = 0x00C0_0000 bit-reversed
00216e20: 0000 0001 0000 0000 0000 0000 0000 0000 ................ | <-- padding for hmac digest #1
00216e30: 0000 0000 0000 0000 0000 0000 00c0 0000 ................ |_______ region hashed for hmac digest #2
00216e40: ae61 607f f1ea 2364 5223 bb1b b7b6 069b .a`...#dR#...... <-- hmac digest #2: SHA256( (hmackey^0x3a | SHA256(bitstream) | SHA_pad) )
00216e50: 2a48 b7f5 dd28 87e0 e10d 3fd0 66e7 cd15 *H...(....?.f...
For some reason, hmac digest #1 area is zeroed out after its computation?
"""
# This is a fixed header for the bitstream, contains sync patterns and configures options such
# as the bitwidth of the config, voltage, etc. This one configures 66MHz config speed, x1, 1.8V
# Making a static copy of this header because it contains commands that modify the operation
# of the FPGA; if we just copy from the incoming bitstream, an attacker may be able to manipulate the
# setup commands in a way that can affect the encryption settings.
# Note: the following pattern is the "bus width" autodetection and is not needed for our design.
# 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
# 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
# 0x00, 0x00, 0x00, 0xBB, 0x11, 0x22, 0x00, 0x44,
bitstream_header_x1 = [
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xaa, 0x99, 0x55, 0x66, 0x20, 0x00, 0x00, 0x00, 0x30, 0x03, 0xe0, 0x01, 0x00, 0x00, 0x00, 0x0b,
0x30, 0x00, 0x80, 0x01, 0x00, 0x00, 0x00, 0x12, 0x20, 0x00, 0x00, 0x00, 0x30, 0x00, 0xc0, 0x01,
0x80, 0x00, 0x00, 0x40, 0x30, 0x00, 0xa0, 0x01, 0x80, 0x00, 0x00, 0x40, 0x30, 0x01, 0xc0, 0x01,
0x00, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x30, 0x01, 0x60, 0x04,
]
# this one selects 66MHz config speed, x2, 1.8V, encryption on, with key from efuse
bitstream_header_x2 = [
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xaa, 0x99, 0x55, 0x66, 0x20, 0x00, 0x00, 0x00, 0x30, 0x03, 0xe0, 0x01, 0x00, 0x00, 0x01, 0x3b,
0x30, 0x00, 0x80, 0x01, 0x00, 0x00, 0x00, 0x12, 0x20, 0x00, 0x00, 0x00, 0x30, 0x00, 0xc0, 0x01,
0x80, 0x00, 0x00, 0x40, 0x30, 0x00, 0xa0, 0x01, 0x80, 0x00, 0x00, 0x40, 0x30, 0x01, 0xc0, 0x01,
0x00, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x30, 0x01, 0x60, 0x04,
]
bitstream_header_bbram_x1 = [
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xaa, 0x99, 0x55, 0x66, 0x20, 0x00, 0x00, 0x00, 0x30, 0x03, 0xe0, 0x01, 0x00, 0x00, 0x00, 0x0b,
0x30, 0x00, 0x80, 0x01, 0x00, 0x00, 0x00, 0x12, 0x20, 0x00, 0x00, 0x00, 0x30, 0x00, 0xc0, 0x01,
0x00, 0x00, 0x00, 0x40, 0x30, 0x00, 0xa0, 0x01, 0x00, 0x00, 0x00, 0x40, 0x30, 0x01, 0xc0, 0x01,
0x00, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x30, 0x01, 0x60, 0x04,
]
# this selects BBRAM for key on a 35T
bitstream_header_35t_bbram_x1 = [
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xAA, 0x99, 0x55, 0x66, 0x20, 0x00, 0x00, 0x00, 0x30, 0x00, 0xC0, 0x01, 0x00, 0x00, 0x00, 0x40,
0x30, 0x00, 0xA0, 0x01, 0x00, 0x00, 0x00, 0x40, 0x30, 0x01, 0xC0, 0x01, 0x00, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
0x20, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x30, 0x01, 0x60, 0x04
]
# then IV key is here 16 bytes
ciphertext_header = [
0x30, 0x03, 0x40, 0x01, 0x00, 0x08, 0x5b, 0x98,
]
"""
Reverse the order of bits in a word that is bitwidth bits wide
"""
def bitflip(data_block, bitwidth=32):
if bitwidth == 0:
return data_block
bytewidth = bitwidth // 8
bitswapped = bytearray()
i = 0
while i < len(data_block):
data = int.from_bytes(data_block[i:i+bytewidth], byteorder='big', signed=False)
b = '{:0{width}b}'.format(data, width=bitwidth)
bitswapped.extend(int(b[::-1], 2).to_bytes(bytewidth, byteorder='big'))
i = i + bytewidth
return bytes(bitswapped)
# assumes a, b are the same length eh?
def xor_bytes(a, b):
i = 0
y = bytearray()
while i < len(a):
y.extend((a[i] ^ b[i]).to_bytes(1, byteorder='little'))
i = i + 1
return bytes(y)
def patcher(bit_in, patch):
bitstream = bit_in[64:-160]
if len(patch) > 0:
# find beginning of type2 area
position = 0
type = -1
command = 0
while type != 2:
command = int.from_bytes(bitflip(bitstream[position:position+4]), byteorder='big')
position = position + 4
if (command & 0xE0000000) == 0x20000000:
type = 1
elif (command & 0xE0000000) == 0x40000000:
type = 2
else:
type = -1
count = 0x3ffffff & command # not used, but handy to have around
ostream = bytearray(bitstream)
# position now sits at the top of the type2 region
# apply patches to each frame specified, ignoring the "none" values
for line in patch:
d = line[1]
for index in range(len(d)):
if d[index].strip() != 'none':
data = bitflip(int(d[index],16).to_bytes(4, 'big'))
frame_num = int(line[0],16)
for b in range(4):
stream_pos = position + frame_num * 101 * 4 + index * 4 + b
ostream[stream_pos] = data[b]
return bytes(ostream)
else:
return bitstream
def dumpframes(ofile, framestream):
position = 0
type = -1
command = 0
while type != 2:
command = int.from_bytes(framestream[position:position+4], byteorder='big')
position = position + 4
if (command & 0xE0000000) == 0x20000000:
type = 1
elif (command & 0xE0000000) == 0x40000000:
type = 2
else:
type = -1
count = 0x3ffffff & command
end = position + count
framecount = 0
while position < end:
ofile.write('0x{:08x},'.format(framecount))
framecount = framecount + 1
for i in range(101):
command = int.from_bytes(framestream[position:position + 4], byteorder='big')
position = position + 4
ofile.write(' 0x{:08x},'.format(command))
ofile.write('\n')
def try_key_to_bytes(input):
if len(input.split(' ')) == 24: # 24 words is BIP-39
# Get if a mnemonic is valid with automatic language detection, return bool
assert(Bip39MnemonicValidator().IsValid(input))
# Like before with automatic language detection
key_bytes = Bip39MnemonicDecoder().Decode(input)
else:
key_bytes = int(input, 16).to_bytes(32, byteorder='big')
return key_bytes
def main():
parser = argparse.ArgumentParser(description="Re-encrypt 7-series bitstream with a new key")
parser.add_argument(
"-f", "--file", required=True, help="Input filename", type=str
)
parser.add_argument(
"-i", "--input-key", required=True, help="Input key as hex string (big endian, without 0x prefix) or BIP-39 format (as quoted string)", type=str
)
parser.add_argument(
"--hmac", required=False, help="Optional HMAC to use for encrypting bitstreams, as hex string. If not provided, a random one is chosen.", type=str
)
parser.add_argument(
"--iv", required=False, help="Optional IV to use for encrypting bitstreams, as hex string. If not provided, a random one is chosen.", type=str
)
parser.add_argument(
"-k", "--key", help="Output key in hex or BIP-39 format. If not provided, plaintext is dumped.", type=str
)
parser.add_argument(
"-b", "--bbram", help="direct FPGA to fetch decryption key from BBRAM", default=False, action="store_true"
)
parser.add_argument(
"-o", "--output-file", help="Output filename base. Generates a .bin and .nky file.", type=str
)
parser.add_argument(
"-d", "--debug", help="turn on debugging spew", default=False, action="store_true"
)
parser.add_argument(
"-a", "--ascii-frame-file", help="dump ascii frames to this file", type=str
)
parser.add_argument(
"-p", "--patch", help="file containing patch frames, as output by key2bits.py", type=str
)
parser.add_argument(
"-j", "--jtag-commands", help="Make JTAG command sequence for BBRAM programming via jtag-gpio.py", default=False, action="store_true"
)
parser.add_argument(
"-t", "--type", choices=['xc7s50', 'xc7a35t'], default='xc7s50', help="select device target", type=str
)
args = parser.parse_args()
ifile = args.file
ofile = Path(args.output_file).stem # strip any suffix off because we create both .bin and .nky
framefile = args.ascii_frame_file
if args.debug:
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
patch = []
if args.patch != None:
patchfile = args.patch
with open(patchfile, "r") as pf:
for lines in pf:
line = lines.split(',')
patch += [[line[0], line[1:]]]
# note: patches must be ordered by frame offset
# extract the keys
key_bytes = try_key_to_bytes(args.input_key)
logging.debug("input key: 0x{}".format(key_bytes.hex()))
if args.key is not None:
new_key = try_key_to_bytes(args.key)
logging.debug("output key: 0x{}".format(new_key.hex()))
else:
new_key = None
if args.hmac is not None:
new_hmac = try_key_to_bytes(args.hmac)
else:
new_hmac = get_random_bytes(32)
if args.iv is not None:
new_iv = try_key_to_bytes(args.iv)
else:
new_iv = get_random_bytes(16)
# format a .nky for the output .bin
if new_key is not None:
with open(ofile + ".nky", "w") as newkey:
if args.type == 'xc7a35t':
newkey.write("Device xc7a35t;\n")
else:
newkey.write("Device xc7s50;\n")
newkey.write("Key 0 ")
newkey.write(new_key.hex())
newkey.write(";\n")
newkey.write("Key StartCBC ")
newkey.write(str(binascii.hexlify(bitflip(new_iv))).strip("b").strip("'"))
newkey.write(";\n")
newkey.write("Key HMAC ")
newkey.write(str(binascii.hexlify(bitflip(new_hmac))).strip("b").strip("'"))
newkey.write(";\n")
if args.jtag_commands:
with open(ofile + ".jtg", "w") as jfile:
jfile.write("ir, 6, 0b010010, program_key\n")
jfile.write("dr, 32, 0xffffffff\n")
jfile.write("ir, 6, 0b010001, isc_program\n")
jfile.write("dr, 32, 0x557b\n")
for index in range(0, 8):
jfile.write("ir, 6, 0b010001, isc_program\n")
jfile.write("dr, 32, 0x{}\n".format(new_key[index*8:(index+1)*8]))
jfile.write("ir, 6, 0b010101, bbkey_rbk\n")
jfile.write("dr, 37, 0x1fffffffff\n")
for index in range(0, 8):
jfile.write("ir, 6, 0b010101, bbkey_rbk\n")
jfile.write("dr, 37, 0x1fffffffff\n")
# open the input file, and recover the plaintext
with open(ifile, "rb") as f:
binfile = f.read()
# search for structure
# 0x3001_6004 -> specifies the CBC key
# 4 words of CBC IV
# 0x3003_4001 -> ciphertext len
# 1 word of ciphertext len
# then ciphertext
position = 0
iv_pos = 0
while position < len(binfile):
cwd = int.from_bytes(binfile[position:position+4], 'big')
if cwd == 0x3001_6004:
iv_pos = position+4
if cwd == 0x3003_4001:
break
position = position + 1
position = position + 4
ciphertext_len = 4* int.from_bytes(binfile[position:position+4], 'big')
logging.debug("ciphertext len: %d", ciphertext_len)
position = position + 4
active_area = binfile[position : position+ciphertext_len]
postamble = binfile[position+ciphertext_len:]
iv_bytes = bitflip(binfile[iv_pos : iv_pos+0x10]) # note that the IV is embedded in the file
logging.debug("recovered iv: %s", binascii.hexlify(iv_bytes))
cipher = AES.new(key_bytes, AES.MODE_CBC, iv_bytes)
plain_bitstream = cipher.decrypt(bitflip(active_area))
# now construct the output file and its hashes
global bitstream_header_x1
global bitstream_header_bbram_x1
global bitstream_header_35t_bbram_x1
plaintext = bytearray()
with open(ofile + ".bin", "wb") as f:
if args.type == 'xc7a35t':
device_header = bitstream_header_35t_bbram_x1
elif args.bbram:
device_header = bitstream_header_bbram_x1
else:
device_header = bitstream_header_x1
for item in device_header: # add the cleartext header
f.write(bytes([item]))
f.write(bitflip(new_iv)) # insert the IV
for item in ciphertext_header: # add the cleartext length-of-ciphertext field before the ciphertext
f.write(bytes([item]))
# generate the header and footer hash keys.
header = int(0x6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C6C).to_bytes(32, byteorder='big')
keyed_header = xor_bytes(header, new_hmac)
footer = int(0x3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A3A).to_bytes(32, byteorder='big')
keyed_footer = xor_bytes(footer, new_hmac)
# add the header
plaintext.extend(keyed_header)
plaintext.extend(header)
# patch the plaintext bitstream, if requested. Also chops the bitstream down to the core size
# Function chops off last 160 bytes, as there will be re-generated with the new HMAC, and header is skipped
plain_patched = patcher(plain_bitstream, patch)
# insert the bitstream plaintext.
plaintext.extend(plain_patched)
# compute first HMAC of stream with new HMAC key
h1 = SHA256.new()
k = 0
while k < len(plaintext) - 320: # HMAC does /not/ cover the whole file, it stops 320 bytes short of the end
h1.update(bitflip(plaintext[k:k+16], 32))
k = k + 16
h1_digest = h1.digest()
logging.debug("new digest1 (in stored order): %s", binascii.hexlify(bitflip(h1_digest)))
# add the footer
plaintext.extend(keyed_footer)
plaintext.extend(footer)
plaintext.extend(bytes(32)) # empty spot where hash #1 would be stored
hash_pad = [ # sha-256 padding for the zero'd hash #1, which is in the bitstream and seems necessary for verification
0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x00, 0x00,
]
plaintext.extend(hash_pad)
# compute the hash of the hash, presumably to prevent length extension attacks?
h2 = SHA256.new()
h2.update(bitflip(keyed_footer))
h2.update(bitflip(footer))
h2.update(h1_digest)
h2_digest = h2.digest()
# commit the final HMAC to the bitstream plaintext
plaintext.extend(bitflip(h2_digest))
logging.debug("new digest2: %s", binascii.hexlify(bitflip(h2_digest)))
if new_key is not None:
logging.debug("new key as used: %s", binascii.hexlify(new_key))
logging.debug("new iv as used: %s", binascii.hexlify(new_iv))
# encrypt the bitstream
newcipher = AES.new(new_key, AES.MODE_CBC, new_iv)
# make a frame dump if requested
if framefile != None:
with open(framefile, "w") as frameoutput:
dumpframes(frameoutput, bitflip(plaintext[64:-160])) # strip off hash header and footer before outputting frames
# finally generate the ciphertext block, which encapsulates the HMACs
ciphertext = newcipher.encrypt(bytes(plaintext))
# add ciphertext to the bitstream
f.write(bitflip(ciphertext))
# add the cleartext postamble to the bitstream. These are a series of NOP commands to the bitstream engine
#postamble = bytearray([0x20, 00, 00, 00])
#for i in range(0,220):
f.write(postamble)
else:
# just dump the plaintext
logging.info("Note: plaintext dump does not have the correct header to be directly loaded into an FPGA (the header would indicate the body is encrypted, when it is not). The purpose of the dump is for debugging encryption/decryption problems, not generating a loadable image.")
# if you wanted to fix the above, we'd have to figure out a bitstream_header record that is commensurate with this purpose. Not hard, just annoying and not needed at this time.
f.write(bitflip(plaintext))
f.write(postamble)
if __name__ == "__main__":
main()