Highly interesting! I kinda wanted not to rely on QEMU as a default "end" for the emulation (As in, I want the end user to be able to choose whatever userspace shim/transport layer/thing they want), but for some of my tests I did forward accesses to QEMU itself (And worked wonders). Thanks for that link! Super cool stuff!
I feel like libfvio-user is a cool project and works perfectly fine, that is, if you want to have the device on the host's userspace but exposed to a VM (QEMU, in this case).
PCIem kinda does that, but it's down a level; in terms of, it basically pops the device on your host PCI bus, which lets real, unmodified drivers to interact with the userspace implementation of your card, no QEMU, no VM, no hypervisors.
Not saying that you can then, for instance, forward all the accesses to QEMU (Some people/orgs already have their cards defined in QEMU so it'd be a bit pointless to redefine the same stuff over and over, right?) so they're free to basically glue their QEMU stuff to PCIem in case they want to try the driver directly on the host but maintaining the functional emulation on QEMU. PCIem takes care of abstracting the accesses and whatnot with an API that tries to mimick that the cool people over at KVM do.
Hi! That's correct. We need a way to have a chunk of what Linux calls "Reserved" memory for the virtual BAR trick. Currently, PCIem only thinks about a single device (Since I first needed to have something that worked in order to check how feasible this all was), but there's planned support for multiple devices that can share a "Reserved" memory pool dynamically so you can have multiple BARs for multiple devices.
Sure! Let's say you (Or the company you work for) are trying to develop an NVME controller card, or a RAID card, or a NIC...
Usually, without actual silicon, you are pretty limited on what you can do in terms of anticipating the software that'll run.
What if you want to write a driver for it w/o having to buy auxiliary boards that act as your card?
What happens if you already have a driver and want to do some security testing on it but don't have the card/don't want to use a physical one for any specific reason (Maybe some UB on the driver pokes at some register that kills the card? Just making disastrous scenarios to prove the point hah).
What if you want to add explicit failures to the card so that you can try and make the driver as tamper-proof and as fault-tolerant as possible (Think, getting the PCI card out of the bus w/o switching the computer off)?
Testing your driver functionally and/or behaviourally on CI/CD on any server (Not requiring the actual card!)?
There's quite a bunch of stuff you can do with it, thanks to being in userspace means that you can get as hacky-wacky as you want (Heck, I have a dumb-framebuffer-esque and OpenGL 1.X capable QEMU device I wanted to write a driver for fun and I used PCIem to forward the accesses to it).
Indeed, the project has gone through a few iterations already (It was first a monolithic kernel module that required a secondary module to call into the API and whatnot). I've went towards a more userspace-friendly usage mainly so that you can iterate your changes much, much faster. Creating the synthetic PCI device is as easy as opening the userspace shim you program, it'll then appear on your bus. When you want to test new changes, you close the shim normally (Effectively removing it from the bus) and you can do this process as many times as needed.
Hi, thanks. That's almost it. The remaining problem is just how to tie it together (where do I put the handle_mmio_read pointer or which event should it be handled in?)
PCIEM_EVENT_MMIO_READ is defined but not used anywhere in the codebase
Hi! Sorry, this is an issue on my side; I forgot to update the documentation's example with the latest changes.
You basically have the kernel eventfd notify you about any access triggered (Based on your configuration), so from userspace, you have the eventfd and then you mmap the shared lock-less ring buffer that actually contains the events PCIem notifies (So you don't end up busy polling).
You basically mmap a struct pciem_shared_ring where you'll have your usual head/tail pointers.
From then on, on your main, you'd have a select() or a poll() for the eventfd; when PCIem notifies the userspace you'd check head != tail (Which means there are events to process) and you can basically do:
The documentation still refers to PCIEM_EVENT_MMIO_READ but it's never referenced in the code on the main branch
I'll admit that I asked for a simple compilable example illustrating something simple like the read events because it looks like it's just reading from the shared memory, and maybe generating an event for any read or write access with the PCIEM_EVENT_MMIO_WRITE event type
PCIEM_EVENT_MMIO_READ is kept for reference on the while (head != tail) loop just to remind the user that there can be more than 1 event registered in terms of access type.
Let's say that you register your watchpoint in READ mode (I still have to change the IOCTL for that as currently is hardcoded for writes: attr.bp_type = HW_BREAKPOINT_W), then you'd be consuming PCIEM_EVENT_MMIO_READ events instead of PCIEM_EVENT_MMIO_WRITE.
The fact that the PCIEM_EVENT_MMIO_READ define is there is to help me remind me to incorporate that missing logic.
Indeed, and even then, there's some sw-hw-codesign stuff that kinda helps you do what PCIem does but it's usually really pricey; so I kinda thought it'd be a good thing to have for free.
PCIe prototyping is usually not something super straightforward if you don't want to pay hefty sums IME.
The "DMA cards" used for video game cheating are generic PCIe cards and (at least the one I got) comes with open documentation (schematics, example projects etc).
Direct Memory Access (DMA) via PCI-e bypasses anti-cheat in the OS because the OS doesn't see the call to read or write the memory. There's no process to spy on, weird drivers, system calls, etc. You can imagine that maybe the anticheat could detect writes that perform a cheat by this method, but it has zero chance of detecting a wallhack style cheat that just reads memory. This is getting to be less relevant with modern OSs, though. Window 11 has IOMMU which only allows DMA to a given memory region defined per device. I think it should be impossible to do this on win11.
