The dummy threads don't "run", they're just thread structs used in place of a real thread. They're essentially a trick used internally during CPU initialization[1] to be able to use the regular context switch to switch into the first thread without having a proper thread context to switch "out" of. There should be protection against situations like you're seeing where the dummy thread is being presented to the scheduler as a real thread struct, thus that assertion.
Basically something has broken your scheduler state somewhere. Is your exerciser something simple you can show against upstream Zephyr? If so, it's probably worth filing an issue in github and tagging or assigning @andyross. If not, can you detail what you're doing so we can try to guess what's going wrong?
Andy
[1] And in one legacy IPC primitive, to be able to have something in a wait queue record that isn't an actual thread. One of these days I hope to fix that, it complicates things in a bunch of areas. But that's unlikely to be your problem unless you're using (and probably modifying the implementation of) kernel mailboxes.