add mjolnir; fix EP allocation bug

Spent the last couple days tracing down a dastardly bug, which was
slew only upon the creation of mjolnir.

The SpinalHDL USB core is unique in that it has a hardware
implementation of a linked list for its descriptors. This can
be very high performance because you can receive packets much
faster than software can keep up with it in short bursts --
assuming the stack can deal with it.

Unfortunately the poor Rust stack that we adopted from
the STM32 land only assumes single descriptors. Alas, we cannot
unlock the full power of linked list descriptors, and instead
we use "single entry lists" with static allocations, to match
the paradigm of the STM32.

Anyways. Early on, when creating the linked list allocator,
I contemplated doing it the Rusty way, and the C way. The Rusty
way would create a structure that mapped the descriptor headers
plus the variable-sized data chunk into some magical structure
that got us bounds checking and had proper accessors etc.
The C way would just be raw pointer manipulation using unsafes.

Unfortunately, my Rust-fu was no match for the linked list.
The problem is that the descriptors could be of any data size,
and they had to be mapped to a very specific piece of memory.
So I had to create a `repr (C)` struct that matched Rust's
alignment requirements and also somehow could deal with
a variable length array *but could not be a slice* because
the hardware implementation does not have the flexibility
to accommodate a fat pointer. I couldn't figure out how
to get const generics to work, and the size of the array
could be anything from 0 bytes to a couple kilobytes. So,
I ditched the safety of Rust and went with a C-style unsafe
pointer implementation.

Sure enough, it came to haunt me. When I wrote the allocator,
I forgot to allocate space for the descriptor headers. This
worked fine for the keyboard, because the keyboard asks for
32-byte descriptors but only ever does a 25-byte report. That's
just enough to fit the descriptor headers, ironically.

When implementing FIDO/U2F, it asks for 64-byte descriptors
and uses exactly 64-byte messages for everything. This caused
a bug where I could receive one incoming packet, but when
responding with an outgoing packet we'd overwrite the descriptors
of the receiving packet, zeroing it out, causing the the descriptor
to fail in the hardware state machine. Of course I never saw
this because I would print the descriptors of the individual
endpoints upon creating them, but the pointer-overflow problem
only occurs later.

Thus mjolnir was created, it is basically a tool that gives me
a raw hex dump of the endpoint and descriptor banks, which I
can then analyze and diff. It's very powerful, but also very
unweildy -- if invoked indiscriminantly, the overhead of
all the prints would cause other failures due to timing problems.
Thus mjolnir exists as a feature that introduces special
locks around the USB endpoints so that they NAK while mjolnir
does its analysis, disallowing concurrent mutation of the
memory state.

Upon deploying mjolnir, the problem became obvious, and it is
now fixed.

Thus, while on the one hand, it's a great story of how maybe
we should avoid unsafe Rust, it's also a story of how hardware
structures like a hardware linked list may be fundamentally
incompatible with Rust. I think if the hardware USB descriptor
linked list was modified to allow for a fat pointer to track
a data slice, the implementation would almost be trivial. But,
rewriting HDL to fit Rust also has its hazards, as the exact
layout of a fat pointer in RAM is not guaranteed and it would
probably foolhardy to commit to something not guaranteed in
hardware.
This commit is contained in:
bunnie 2022-06-03 23:53:37 +08:00
parent 4dca789d22
commit 3ad70c711d
4 changed files with 108 additions and 49 deletions

View file

@ -42,4 +42,5 @@ utralib = { path = "../../utralib"}
[features] [features]
keyboard = [] # handy for debugging composite device issues keyboard = [] # handy for debugging composite device issues
mjolnir = [] # the big hammer for debugging Spinal USB issues. A raw memory dump of config and descriptor space. Use with care.
default = ["keyboard"] default = ["keyboard"]

View file

@ -228,6 +228,29 @@ impl SpinalUsbDevice {
} }
} }
} }
#[cfg(feature="mjolnir")]
pub fn ll_debug(&self) {
let mut regblock = String::new();
let regs = unsafe{self.usb.as_mut_ptr().add(0xFF00) as *const u32};
regblock.push_str("\r\nRegs:\r\n");
for i in 0..16 {
regblock.push_str(
&format!("{:08x}, {:08x},\r\n",
unsafe{regs.add(i*2).read_volatile()},
unsafe{regs.add(i*2 +1).read_volatile()}));
}
log::info!("{}", regblock);
let mut eps = String::from("\r\nEps/descs:");
let d = unsafe{self.usb.as_mut_ptr().add(0) as *const u32};
for i in 0..72 {
if i % 4 == 0 {
eps.push_str("\r\n");
}
eps.push_str(&format!("{:08x}, ", unsafe{d.add(i).read_volatile()}));
}
eps.push_str("\r\n");
log::info!("{}", eps);
}
/// simple but easy to understand allocator for buffers inside the descriptor memory space /// simple but easy to understand allocator for buffers inside the descriptor memory space
/// See notes inside src/main.rs `alloc_inner` for the functional description. Returns /// See notes inside src/main.rs `alloc_inner` for the functional description. Returns
/// the full byte-addressed offset of the region, so it must be shifted to the right by /// the full byte-addressed offset of the region, so it must be shifted to the right by
@ -378,7 +401,7 @@ impl UsbBus for SpinalUsbDevice {
descriptor.set_desc_flags( descriptor.set_desc_flags(
ep_dir, ep_dir,
ep_dir == UsbDirection::Out, ep_dir == UsbDirection::Out,
ep_dir == UsbDirection::Out, // this should be equal to "packet_end", but this driver doesn't have that...? true, // this should be equal to "packet_end", but this driver doesn't have that...?
index == 0, // only trigger for ep0 (per spinal linux driver) index == 0, // only trigger for ep0 (per spinal linux driver)
); );
// clear the descriptor to 0 // clear the descriptor to 0
@ -493,7 +516,8 @@ impl UsbBus for SpinalUsbDevice {
} }
} }
} }
log::debug!("{:?}", self.regs); #[cfg(feature="mjolnir")]
self.ll_debug(); // this is the "memory dump" of the USB descriptor area.
// clear other registers // clear other registers
// self.regs.set_address(0); // i think this is automatic in the USB core... // self.regs.set_address(0); // i think this is automatic in the USB core...
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
@ -530,6 +554,8 @@ impl UsbBus for SpinalUsbDevice {
if buf.len() > max_len { if buf.len() > max_len {
Err(UsbError::BufferOverflow) Err(UsbError::BufferOverflow)
} else { } else {
#[cfg(feature="mjolnir")] // mjolnir is so powerful, one must halt the USB core entirely for it to be weilded
if ep_addr.index() == 1 { self.udc_hard_halt(ep_addr.index()); }
let mut ep_status = self.status_read_volatile(ep_addr.index()); let mut ep_status = self.status_read_volatile(ep_addr.index());
if ep_addr.index() != 0 { if ep_addr.index() != 0 {
if ep_status.head_offset() != 0 { if ep_status.head_offset() != 0 {
@ -573,21 +599,34 @@ impl UsbBus for SpinalUsbDevice {
descriptor.set_next_desc_and_len(0, buf.len()); descriptor.set_next_desc_and_len(0, buf.len());
if ep_addr.index() != 0 { if ep_addr.index() != 0 {
descriptor.set_desc_flags(UsbDirection::In, descriptor.set_desc_flags(UsbDirection::In,
false, false, false); false, true, false);
} }
descriptor.set_offset(0); // reset the write pointer to 0, also sets in_progress descriptor.set_offset(0); // reset the write pointer to 0, also sets in_progress
if ep_addr.index() != 0 { //if ep_addr.index() != 0 {
log::info!("WR PREdesc{}: {:?}", ep_addr.index(), descriptor); // log::info!("WR PREdesc{}: {:?}", ep_addr.index(), descriptor);
} //}
// this is required to commit the ep_status record once all the setup is done // this is required to commit the ep_status record once all the setup is done
self.status_write_volatile(ep_addr.index(), ep_status); self.status_write_volatile(ep_addr.index(), ep_status);
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
let epcheck = self.status_read_volatile(ep_addr.index()); #[cfg(feature="mjolnir")]
let epcheckdesc = self.descriptor_from_status(&epcheck); { // mjolnir is so powerful, one must halt the USB core entirely for it to be weilded
if ep_addr.index() != 0 { if ep_addr.index() == 1 {
log::info!("WR status{}: {:?}", ep_addr.index(), epcheck); self.ll_debug();
log::info!("WR desc{}: {:?}", ep_addr.index(), epcheckdesc); }
if ep_addr.index() == 1 { self.udc_hard_unhalt(ep_addr.index()); }
} }
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
// I think the debug statements actually mess up the timing of the core (as it causes contention for the same
// memory resources used by the core itself), so they are commented out.
//let epcheck = self.status_read_volatile(ep_addr.index());
//let epcheckdesc = self.descriptor_from_status(&epcheck);
//if ep_addr.index() != 0 {
// log::info!("WR status{}: {:?}", ep_addr.index(), epcheck);
// log::info!("WR desc{}: {:?}", ep_addr.index(), epcheckdesc);
// log::info!("WR POSTstatus{}: {:?}", ep_addr.index(), self.status_read_volatile(ep_addr.index()));
//}
Ok(buf.len()) Ok(buf.len())
} }
} else { } else {
@ -662,7 +701,8 @@ impl UsbBus for SpinalUsbDevice {
// get to the bottom of it. But for now it seems to work well enough and the performance is "fine" for // get to the bottom of it. But for now it seems to work well enough and the performance is "fine" for
// a USB HID style interface. // a USB HID style interface.
// //
// N.B.: I *think* this got solved when we also solved the spurious read interrupt issue. // I thought this was fixed by removing spurious interrupts, but instead, it seems to come back "sometimes",
// as opposed to "always" being a problem. So the delay is re-instated.
self.tt.sleep_ms(1).ok(); self.tt.sleep_ms(1).ok();
if (self.read_allowed.load(Ordering::Relaxed) & (1 << ep_addr.index() as u16)) == 0 { if (self.read_allowed.load(Ordering::Relaxed) & (1 << ep_addr.index() as u16)) == 0 {
@ -676,7 +716,7 @@ impl UsbBus for SpinalUsbDevice {
self.read_allowed.store(masked, Ordering::SeqCst); self.read_allowed.store(masked, Ordering::SeqCst);
} }
// self.udc_hard_halt(ep_addr.index()); self.udc_hard_halt(ep_addr.index());
let mut ep_status = self.status_read_volatile(ep_addr.index()); let mut ep_status = self.status_read_volatile(ep_addr.index());
// log::info!("head_offset{}: {:x}", ep_addr.index(), head_offset * 16); // log::info!("head_offset{}: {:x}", ep_addr.index(), head_offset * 16);
ep_status.set_head_offset(head_offset as u32); ep_status.set_head_offset(head_offset as u32);
@ -709,30 +749,38 @@ impl UsbBus for SpinalUsbDevice {
buf[(len / 4) + i] = word[i] buf[(len / 4) + i] = word[i]
} }
} }
// setup for the next transaction // setup for the next transaction
ep_status.set_max_packet_size(max_len as _); ep_status.set_max_packet_size(max_len as _);
descriptor.set_next_desc_and_len(0, buf.len()); descriptor.set_next_desc_and_len(0, buf.len());
descriptor.set_desc_flags(UsbDirection::Out, descriptor.set_desc_flags(UsbDirection::Out,
true, true, false); true, true, false);
descriptor.set_offset_only(0); // reset the read pointer to 0, also sets in_progress
// sanity check on interrupts: it should be cleared by this point!
let interrupts = self.regs.interrupts();
if interrupts.0 != 0 {
log::warn!("Pending interrupts, clearing: {:?}", interrupts);
self.regs.clear_all_interrupts();
}
// this auto-toggles ep_status.set_data_phase(!ep_status.data_phase()); // toggle the data phase // this auto-toggles ep_status.set_data_phase(!ep_status.data_phase()); // toggle the data phase
self.status_write_volatile(ep_addr.index(), ep_status); self.status_write_volatile(ep_addr.index(), ep_status);
descriptor.set_offset(0); // reset the read pointer to 0, also sets in_progress core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
// if we happen to pick this read up before the interrupt handler can fire, clear the interrupt // just reading the registers after setting them seems to upset the USB device block
let interrupts = self.regs.interrupts(); //log::debug!("read buf: {:x?}", &buf[..len]);
if interrupts.endpoint() & (1 << ep_addr.index()) != 0 { //log::info!("ep{} read: {:x?} (len {} into buf of {})", ep_addr.index(), &buf[..len], len, buf.len());
log::warn!("read happened before interrupt handler could process"); let epcheck = self.status_read_volatile(ep_addr.index());
/* let mut ints_to_clear = UdcInterrupts(0); let descheck = self.descriptor_from_status(&epcheck);
ints_to_clear.set_endpoint(1 << ep_addr.index()); log::info!("RD status{} [{:x}]: {:?}", ep_addr.index(), epcheck.0, epcheck);
self.regs.clear_some_interrupts(ints_to_clear); */ log::info!("RD desc{} [{:x},{:x},{:x}]: {:?}", ep_addr.index(), descheck.read(0), descheck.read(1), descheck.read(2), descheck);
#[cfg(feature="mjolnir")]
if ep_addr.index() == 2 { // mjolnir should be use selectively on EPs that require debugging, lest the weilder be overwhelmed by its spew.
self.ll_debug();
} }
// self.udc_hard_unhalt(ep_addr.index()); self.udc_hard_unhalt(ep_addr.index());
core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst);
log::debug!("read buf: {:x?}", &buf[..len]);
log::info!("ep{} read: {:x?} (len {} into buf of {})", ep_addr.index(), &buf[..len], len, buf.len());
log::info!("RD status{}: {:?}", ep_addr.index(), self.status_read_volatile(ep_addr.index()));
log::info!("RD desc{}: {:?}", ep_addr.index(), self.descriptor_from_status(&self.status_read_volatile(ep_addr.index())));
Ok(len) Ok(len)
} }
} else { } else {
@ -752,23 +800,28 @@ impl UsbBus for SpinalUsbDevice {
//if ep_addr.index() == 0 && ep_addr.direction() == UsbDirection::Out && stalled == false { //if ep_addr.index() == 0 && ep_addr.direction() == UsbDirection::Out && stalled == false {
// return; // return;
//} //}
log::debug!("set_stalled ep{}->{} dir {:?}", ep_addr.index(), stalled, ep_addr.direction()); let statcheck = self.status_read_volatile(ep_addr.index());
self.udc_hard_halt(ep_addr.index()); if statcheck.force_stall() != stalled {
let mut ep_status = self.status_read_volatile(ep_addr.index()); if ep_addr.index() != 0 {
match (stalled, ep_addr.direction()) { log::info!("set_stalled ep{}->{} dir {:?}", ep_addr.index(), stalled, ep_addr.direction());
(true, UsbDirection::In) => { }
ep_status.set_force_stall(true); self.udc_hard_halt(ep_addr.index());
}, let mut ep_status = self.status_read_volatile(ep_addr.index());
(true, UsbDirection::Out) => ep_status.set_force_stall(true), match (stalled, ep_addr.direction()) {
(false, UsbDirection::In) => { (true, UsbDirection::In) => {
ep_status.set_force_stall(false); ep_status.set_force_stall(true);
ep_status.set_force_nack(true); },
}, (true, UsbDirection::Out) => ep_status.set_force_stall(true),
(false, UsbDirection::Out) => ep_status.set_force_stall(false), (false, UsbDirection::In) => {
}; ep_status.set_force_stall(false);
// single volatile commit of the results ep_status.set_force_nack(true);
self.status_write_volatile(ep_addr.index(), ep_status); },
self.udc_hard_unhalt(ep_addr.index()); (false, UsbDirection::Out) => ep_status.set_force_stall(false),
};
// single volatile commit of the results
self.status_write_volatile(ep_addr.index(), ep_status);
self.udc_hard_unhalt(ep_addr.index());
}
} }
/// Gets whether the STALL condition is set for an endpoint. /// Gets whether the STALL condition is set for an endpoint.
@ -806,6 +859,7 @@ impl UsbBus for SpinalUsbDevice {
PollResult::Reset PollResult::Reset
} else if interrupts.ep0_setup() { } else if interrupts.ep0_setup() {
ints_to_clear.set_ep0_setup(true); ints_to_clear.set_ep0_setup(true);
ints_to_clear.set_endpoint(1); // this is a bitmask, so this corresponds to ep0
PollResult::Data { ep_out: 0, ep_in_complete: 0, ep_setup: 1 } PollResult::Data { ep_out: 0, ep_in_complete: 0, ep_setup: 1 }
} else if interrupts.endpoint() != 0 { } else if interrupts.endpoint() != 0 {
let mut ep_in_complete = 0; let mut ep_in_complete = 0;
@ -823,6 +877,9 @@ impl UsbBus for SpinalUsbDevice {
// it will get written back on the next `write`) // it will get written back on the next `write`)
ep_status.set_head_offset(self.ep0in_head); ep_status.set_head_offset(self.ep0in_head);
} else if let Some((head_offset, _max_len)) = self.ep_allocs[bit] { } else if let Some((head_offset, _max_len)) = self.ep_allocs[bit] {
if ep_status.head_offset() != 0 {
log::warn!("got INT on ep{} but head is not 0", bit);
}
ep_status.set_head_offset(head_offset as u32); ep_status.set_head_offset(head_offset as u32);
} }
let descriptor = self.descriptor_from_status(&ep_status); let descriptor = self.descriptor_from_status(&ep_status);

View file

@ -463,6 +463,7 @@ pub(crate) fn alloc_inner(allocs: &mut BTreeMap<u32, u32>, requested: u32) -> Op
if requested == 0 { if requested == 0 {
return None; return None;
} }
let with_descriptor = requested + 16; // the descriptor takes 3 words; add 4 because of the alignment requirement
let mut alloc_offset = START_OFFSET; let mut alloc_offset = START_OFFSET;
for (&offset, &length) in allocs.iter() { for (&offset, &length) in allocs.iter() {
// round length up to the nearest 16-byte increment // round length up to the nearest 16-byte increment
@ -470,15 +471,15 @@ pub(crate) fn alloc_inner(allocs: &mut BTreeMap<u32, u32>, requested: u32) -> Op
// println!("aoff: {}, cur: {}+{}", alloc_offset, offset, length); // println!("aoff: {}, cur: {}+{}", alloc_offset, offset, length);
assert!(offset >= alloc_offset, "allocated regions overlap"); assert!(offset >= alloc_offset, "allocated regions overlap");
if offset > alloc_offset { if offset > alloc_offset {
if offset - alloc_offset >= requested { if offset - alloc_offset >= with_descriptor {
// there's a hole in the list, insert the element here // there's a hole in the list, insert the element here
break; break;
} }
} }
alloc_offset = offset + length; alloc_offset = offset + length;
} }
if alloc_offset + requested <= END_OFFSET { if alloc_offset + with_descriptor <= END_OFFSET {
allocs.insert(alloc_offset, requested); allocs.insert(alloc_offset, with_descriptor);
Some(alloc_offset) Some(alloc_offset)
} else { } else {
None None

View file

@ -235,7 +235,7 @@ impl SpinalUdcDescriptor {
pub fn new(base: *mut u32) -> SpinalUdcDescriptor { pub fn new(base: *mut u32) -> SpinalUdcDescriptor {
SpinalUdcDescriptor { base: AtomicPtr::new(base) } SpinalUdcDescriptor { base: AtomicPtr::new(base) }
} }
fn read(&self, offset: usize) -> u32 { pub(crate) fn read(&self, offset: usize) -> u32 {
// we don't do asserts on reads because for debugging sometimes we reveal invalid descriptors and that's OK // we don't do asserts on reads because for debugging sometimes we reveal invalid descriptors and that's OK
unsafe{self.base.load(Ordering::SeqCst).add(offset).read_volatile()} unsafe{self.base.load(Ordering::SeqCst).add(offset).read_volatile()}
} }