add mjolnir; fix EP allocation bug

Spent the last couple days tracing down a dastardly bug, which was
slew only upon the creation of mjolnir.

The SpinalHDL USB core is unique in that it has a hardware
implementation of a linked list for its descriptors. This can
be very high performance because you can receive packets much
faster than software can keep up with it in short bursts --
assuming the stack can deal with it.

Unfortunately the poor Rust stack that we adopted from
the STM32 land only assumes single descriptors. Alas, we cannot
unlock the full power of linked list descriptors, and instead
we use "single entry lists" with static allocations, to match
the paradigm of the STM32.

Anyways. Early on, when creating the linked list allocator,
I contemplated doing it the Rusty way, and the C way. The Rusty
way would create a structure that mapped the descriptor headers
plus the variable-sized data chunk into some magical structure
that got us bounds checking and had proper accessors etc.
The C way would just be raw pointer manipulation using unsafes.

Unfortunately, my Rust-fu was no match for the linked list.
The problem is that the descriptors could be of any data size,
and they had to be mapped to a very specific piece of memory.
So I had to create a `repr (C)` struct that matched Rust's
alignment requirements and also somehow could deal with
a variable length array *but could not be a slice* because
the hardware implementation does not have the flexibility
to accommodate a fat pointer. I couldn't figure out how
to get const generics to work, and the size of the array
could be anything from 0 bytes to a couple kilobytes. So,
I ditched the safety of Rust and went with a C-style unsafe
pointer implementation.

Sure enough, it came to haunt me. When I wrote the allocator,
I forgot to allocate space for the descriptor headers. This
worked fine for the keyboard, because the keyboard asks for
32-byte descriptors but only ever does a 25-byte report. That's
just enough to fit the descriptor headers, ironically.

When implementing FIDO/U2F, it asks for 64-byte descriptors
and uses exactly 64-byte messages for everything. This caused
a bug where I could receive one incoming packet, but when
responding with an outgoing packet we'd overwrite the descriptors
of the receiving packet, zeroing it out, causing the the descriptor
to fail in the hardware state machine. Of course I never saw
this because I would print the descriptors of the individual
endpoints upon creating them, but the pointer-overflow problem
only occurs later.

Thus mjolnir was created, it is basically a tool that gives me
a raw hex dump of the endpoint and descriptor banks, which I
can then analyze and diff. It's very powerful, but also very
unweildy -- if invoked indiscriminantly, the overhead of
all the prints would cause other failures due to timing problems.
Thus mjolnir exists as a feature that introduces special
locks around the USB endpoints so that they NAK while mjolnir
does its analysis, disallowing concurrent mutation of the
memory state.

Upon deploying mjolnir, the problem became obvious, and it is
now fixed.

Thus, while on the one hand, it's a great story of how maybe
we should avoid unsafe Rust, it's also a story of how hardware
structures like a hardware linked list may be fundamentally
incompatible with Rust. I think if the hardware USB descriptor
linked list was modified to allow for a fat pointer to track
a data slice, the implementation would almost be trivial. But,
rewriting HDL to fit Rust also has its hazards, as the exact
layout of a fat pointer in RAM is not guaranteed and it would
probably foolhardy to commit to something not guaranteed in
hardware.
This commit is contained in:
bunnie 2022-06-03 23:53:37 +08:00
parent 4dca789d22
commit 3ad70c711d
4 changed files with 108 additions and 49 deletions

View file

@ -42,4 +42,5 @@ utralib = { path = "../../utralib"}
[features]
keyboard = [] # handy for debugging composite device issues
mjolnir = [] # the big hammer for debugging Spinal USB issues. A raw memory dump of config and descriptor space. Use with care.
default = ["keyboard"]

View file

@ -228,6 +228,29 @@ impl SpinalUsbDevice {
}
}
}
#[cfg(feature="mjolnir")]
pub fn ll_debug(&self) {
let mut regblock = String::new();
let regs = unsafe{self.usb.as_mut_ptr().add(0xFF00) as *const u32};
regblock.push_str("\r\nRegs:\r\n");
for i in 0..16 {
regblock.push_str(
&format!("{:08x}, {:08x},\r\n",
unsafe{regs.add(i*2).read_volatile()},
unsafe{regs.add(i*2 +1).read_volatile()}));
}
log::info!("{}", regblock);
let mut eps = String::from("\r\nEps/descs:");
let d = unsafe{self.usb.as_mut_ptr().add(0) as *const u32};
for i in 0..72 {
if i % 4 == 0 {
eps.push_str("\r\n");
}
eps.push_str(&format!("{:08x}, ", unsafe{d.add(i).read_volatile()}));
}
eps.push_str("\r\n");
log::info!("{}", eps);
}
/// simple but easy to understand allocator for buffers inside the descriptor memory space
/// See notes inside src/main.rs `alloc_inner` for the functional description. Returns
/// the full byte-addressed offset of the region, so it must be shifted to the right by
@ -378,7 +401,7 @@ impl UsbBus for SpinalUsbDevice {
descriptor.set_desc_flags(
ep_dir,
ep_dir == UsbDirection::Out,
ep_dir == UsbDirection::Out, // this should be equal to "packet_end", but this driver doesn't have that...?
true, // this should be equal to "packet_end", but this driver doesn't have that...?
index == 0, // only trigger for ep0 (per spinal linux driver)
);
// clear the descriptor to 0
@ -493,7 +516,8 @@ impl UsbBus for SpinalUsbDevice {
}
}
}
log::debug!("{:?}", self.regs);
#[cfg(feature="mjolnir")]
self.ll_debug(); // this is the "memory dump" of the USB descriptor area.
// clear other registers
// self.regs.set_address(0); // i think this is automatic in the USB core...
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
@ -530,6 +554,8 @@ impl UsbBus for SpinalUsbDevice {
if buf.len() > max_len {
Err(UsbError::BufferOverflow)
} else {
#[cfg(feature="mjolnir")] // mjolnir is so powerful, one must halt the USB core entirely for it to be weilded
if ep_addr.index() == 1 { self.udc_hard_halt(ep_addr.index()); }
let mut ep_status = self.status_read_volatile(ep_addr.index());
if ep_addr.index() != 0 {
if ep_status.head_offset() != 0 {
@ -573,21 +599,34 @@ impl UsbBus for SpinalUsbDevice {
descriptor.set_next_desc_and_len(0, buf.len());
if ep_addr.index() != 0 {
descriptor.set_desc_flags(UsbDirection::In,
false, false, false);
false, true, false);
}
descriptor.set_offset(0); // reset the write pointer to 0, also sets in_progress
if ep_addr.index() != 0 {
log::info!("WR PREdesc{}: {:?}", ep_addr.index(), descriptor);
}
//if ep_addr.index() != 0 {
// log::info!("WR PREdesc{}: {:?}", ep_addr.index(), descriptor);
//}
// this is required to commit the ep_status record once all the setup is done
self.status_write_volatile(ep_addr.index(), ep_status);
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
let epcheck = self.status_read_volatile(ep_addr.index());
let epcheckdesc = self.descriptor_from_status(&epcheck);
if ep_addr.index() != 0 {
log::info!("WR status{}: {:?}", ep_addr.index(), epcheck);
log::info!("WR desc{}: {:?}", ep_addr.index(), epcheckdesc);
#[cfg(feature="mjolnir")]
{ // mjolnir is so powerful, one must halt the USB core entirely for it to be weilded
if ep_addr.index() == 1 {
self.ll_debug();
}
if ep_addr.index() == 1 { self.udc_hard_unhalt(ep_addr.index()); }
}
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
// I think the debug statements actually mess up the timing of the core (as it causes contention for the same
// memory resources used by the core itself), so they are commented out.
//let epcheck = self.status_read_volatile(ep_addr.index());
//let epcheckdesc = self.descriptor_from_status(&epcheck);
//if ep_addr.index() != 0 {
// log::info!("WR status{}: {:?}", ep_addr.index(), epcheck);
// log::info!("WR desc{}: {:?}", ep_addr.index(), epcheckdesc);
// log::info!("WR POSTstatus{}: {:?}", ep_addr.index(), self.status_read_volatile(ep_addr.index()));
//}
Ok(buf.len())
}
} else {
@ -662,7 +701,8 @@ impl UsbBus for SpinalUsbDevice {
// get to the bottom of it. But for now it seems to work well enough and the performance is "fine" for
// a USB HID style interface.
//
// N.B.: I *think* this got solved when we also solved the spurious read interrupt issue.
// I thought this was fixed by removing spurious interrupts, but instead, it seems to come back "sometimes",
// as opposed to "always" being a problem. So the delay is re-instated.
self.tt.sleep_ms(1).ok();
if (self.read_allowed.load(Ordering::Relaxed) & (1 << ep_addr.index() as u16)) == 0 {
@ -676,7 +716,7 @@ impl UsbBus for SpinalUsbDevice {
self.read_allowed.store(masked, Ordering::SeqCst);
}
// self.udc_hard_halt(ep_addr.index());
self.udc_hard_halt(ep_addr.index());
let mut ep_status = self.status_read_volatile(ep_addr.index());
// log::info!("head_offset{}: {:x}", ep_addr.index(), head_offset * 16);
ep_status.set_head_offset(head_offset as u32);
@ -709,30 +749,38 @@ impl UsbBus for SpinalUsbDevice {
buf[(len / 4) + i] = word[i]
}
}
// setup for the next transaction
ep_status.set_max_packet_size(max_len as _);
descriptor.set_next_desc_and_len(0, buf.len());
descriptor.set_desc_flags(UsbDirection::Out,
true, true, false);
descriptor.set_offset_only(0); // reset the read pointer to 0, also sets in_progress
// sanity check on interrupts: it should be cleared by this point!
let interrupts = self.regs.interrupts();
if interrupts.0 != 0 {
log::warn!("Pending interrupts, clearing: {:?}", interrupts);
self.regs.clear_all_interrupts();
}
// this auto-toggles ep_status.set_data_phase(!ep_status.data_phase()); // toggle the data phase
self.status_write_volatile(ep_addr.index(), ep_status);
descriptor.set_offset(0); // reset the read pointer to 0, also sets in_progress
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
// if we happen to pick this read up before the interrupt handler can fire, clear the interrupt
let interrupts = self.regs.interrupts();
if interrupts.endpoint() & (1 << ep_addr.index()) != 0 {
log::warn!("read happened before interrupt handler could process");
/* let mut ints_to_clear = UdcInterrupts(0);
ints_to_clear.set_endpoint(1 << ep_addr.index());
self.regs.clear_some_interrupts(ints_to_clear); */
// just reading the registers after setting them seems to upset the USB device block
//log::debug!("read buf: {:x?}", &buf[..len]);
//log::info!("ep{} read: {:x?} (len {} into buf of {})", ep_addr.index(), &buf[..len], len, buf.len());
let epcheck = self.status_read_volatile(ep_addr.index());
let descheck = self.descriptor_from_status(&epcheck);
log::info!("RD status{} [{:x}]: {:?}", ep_addr.index(), epcheck.0, epcheck);
log::info!("RD desc{} [{:x},{:x},{:x}]: {:?}", ep_addr.index(), descheck.read(0), descheck.read(1), descheck.read(2), descheck);
#[cfg(feature="mjolnir")]
if ep_addr.index() == 2 { // mjolnir should be use selectively on EPs that require debugging, lest the weilder be overwhelmed by its spew.
self.ll_debug();
}
// self.udc_hard_unhalt(ep_addr.index());
log::debug!("read buf: {:x?}", &buf[..len]);
log::info!("ep{} read: {:x?} (len {} into buf of {})", ep_addr.index(), &buf[..len], len, buf.len());
log::info!("RD status{}: {:?}", ep_addr.index(), self.status_read_volatile(ep_addr.index()));
log::info!("RD desc{}: {:?}", ep_addr.index(), self.descriptor_from_status(&self.status_read_volatile(ep_addr.index())));
self.udc_hard_unhalt(ep_addr.index());
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
Ok(len)
}
} else {
@ -752,23 +800,28 @@ impl UsbBus for SpinalUsbDevice {
//if ep_addr.index() == 0 && ep_addr.direction() == UsbDirection::Out && stalled == false {
// return;
//}
log::debug!("set_stalled ep{}->{} dir {:?}", ep_addr.index(), stalled, ep_addr.direction());
self.udc_hard_halt(ep_addr.index());
let mut ep_status = self.status_read_volatile(ep_addr.index());
match (stalled, ep_addr.direction()) {
(true, UsbDirection::In) => {
ep_status.set_force_stall(true);
},
(true, UsbDirection::Out) => ep_status.set_force_stall(true),
(false, UsbDirection::In) => {
ep_status.set_force_stall(false);
ep_status.set_force_nack(true);
},
(false, UsbDirection::Out) => ep_status.set_force_stall(false),
};
// single volatile commit of the results
self.status_write_volatile(ep_addr.index(), ep_status);
self.udc_hard_unhalt(ep_addr.index());
let statcheck = self.status_read_volatile(ep_addr.index());
if statcheck.force_stall() != stalled {
if ep_addr.index() != 0 {
log::info!("set_stalled ep{}->{} dir {:?}", ep_addr.index(), stalled, ep_addr.direction());
}
self.udc_hard_halt(ep_addr.index());
let mut ep_status = self.status_read_volatile(ep_addr.index());
match (stalled, ep_addr.direction()) {
(true, UsbDirection::In) => {
ep_status.set_force_stall(true);
},
(true, UsbDirection::Out) => ep_status.set_force_stall(true),
(false, UsbDirection::In) => {
ep_status.set_force_stall(false);
ep_status.set_force_nack(true);
},
(false, UsbDirection::Out) => ep_status.set_force_stall(false),
};
// single volatile commit of the results
self.status_write_volatile(ep_addr.index(), ep_status);
self.udc_hard_unhalt(ep_addr.index());
}
}
/// Gets whether the STALL condition is set for an endpoint.
@ -806,6 +859,7 @@ impl UsbBus for SpinalUsbDevice {
PollResult::Reset
} else if interrupts.ep0_setup() {
ints_to_clear.set_ep0_setup(true);
ints_to_clear.set_endpoint(1); // this is a bitmask, so this corresponds to ep0
PollResult::Data { ep_out: 0, ep_in_complete: 0, ep_setup: 1 }
} else if interrupts.endpoint() != 0 {
let mut ep_in_complete = 0;
@ -823,6 +877,9 @@ impl UsbBus for SpinalUsbDevice {
// it will get written back on the next `write`)
ep_status.set_head_offset(self.ep0in_head);
} else if let Some((head_offset, _max_len)) = self.ep_allocs[bit] {
if ep_status.head_offset() != 0 {
log::warn!("got INT on ep{} but head is not 0", bit);
}
ep_status.set_head_offset(head_offset as u32);
}
let descriptor = self.descriptor_from_status(&ep_status);

View file

@ -463,6 +463,7 @@ pub(crate) fn alloc_inner(allocs: &mut BTreeMap<u32, u32>, requested: u32) -> Op
if requested == 0 {
return None;
}
let with_descriptor = requested + 16; // the descriptor takes 3 words; add 4 because of the alignment requirement
let mut alloc_offset = START_OFFSET;
for (&offset, &length) in allocs.iter() {
// round length up to the nearest 16-byte increment
@ -470,15 +471,15 @@ pub(crate) fn alloc_inner(allocs: &mut BTreeMap<u32, u32>, requested: u32) -> Op
// println!("aoff: {}, cur: {}+{}", alloc_offset, offset, length);
assert!(offset >= alloc_offset, "allocated regions overlap");
if offset > alloc_offset {
if offset - alloc_offset >= requested {
if offset - alloc_offset >= with_descriptor {
// there's a hole in the list, insert the element here
break;
}
}
alloc_offset = offset + length;
}
if alloc_offset + requested <= END_OFFSET {
allocs.insert(alloc_offset, requested);
if alloc_offset + with_descriptor <= END_OFFSET {
allocs.insert(alloc_offset, with_descriptor);
Some(alloc_offset)
} else {
None

View file

@ -235,7 +235,7 @@ impl SpinalUdcDescriptor {
pub fn new(base: *mut u32) -> SpinalUdcDescriptor {
SpinalUdcDescriptor { base: AtomicPtr::new(base) }
}
fn read(&self, offset: usize) -> u32 {
pub(crate) fn read(&self, offset: usize) -> u32 {
// we don't do asserts on reads because for debugging sometimes we reveal invalid descriptors and that's OK
unsafe{self.base.load(Ordering::SeqCst).add(offset).read_volatile()}
}