Awesome, thanks! This is exactly the kind of experienced take I was hoping my blog post would summon =D
Re: computing M and s, does torch.quantization.quantize_qat do this or do you do it yourself from the (presumably f32) activation scaling that torch finds?
I don't have much experience with this kind of numerical computing, so I have no intuition about how much the "quantization" of selecting M and s might impact the overall performance of the network. I.e., whether
- M and s should be trained as part of QAT (e.g., the "Learned Step Size Quantization" paper)
- it's fine to just deterministically compute M and s from the f32 activation scaling.
Also: Thanks for the tips re: CMSIS-NN, glad to know it's possible to use in a non-framework way. Any chance your example is open source somewhere?
For my application I need just the translations and Euler angles. The range of poses is mechanically constrained so I don't have to worry about gimbal lock. But yeah, my limited understanding matches yours that other parameterizations are more useful in general contexts.
Hey there op. I don't know what your sensors are measuring (distance to a point maybe? Or angle from a Valve lighthouse for inside-out tracking?)
But here's my "why didn't you just"
Since you have a forward simulation function (pose to measurements), why didn't you use an iterative solver to reverse it? Coordinate descent is easy to code and if you have a constrained range of poses you can probably just use multiple starting points to avoid getting stuck with a local minimum. Then use the last solution as a starting point for the next one to save iterations.
Sure it's not closed-form like an NN and it can still have pathological cases, but the code is a little more transparent
That's a reasonable idea, but unfortunately wouldn't work in my case since the simulation relies on a lot of scientific libraries in Python and I need the inversion to happen on the microcontroller.
When you say "coordinate descent" do you mean gradient descent? I.e., updating a potential pose using the gradient of a loss term (e.g., (predicted sensor reading - actual sensor reading)**2)?
I bet that would work, but a tricky part would be calculating gradients. I'm not sure if the Python libraries I'm using support that. My understanding is that automatic differentiation through libraries might be easier in a language like Julia where dual numbers flow through everything via the multiple dispatch mechanism.
Author here. Lots of questions about precision --- in the article I calculated 0.6mm as the standard deviation of 200ms worth of phase measurements (n=124) while the slide wasn't moving.
I'd love to hear any advice/ideas re:
- approaches for figuring out what's currently limiting the accuracy (i.e., noise sources)
- the relative merits of averaging in time vs phase domain
I kind of glossed over your article but one thing that stood out is that you preferred to lower the adc sample rate. That doesn't make sense.
Usually the way you want to set something like this up is with a low pass filter before you sample it and you want to sample at a high enough rate (some multiple of that filter frequency). This is due to the sampling theorem. Maybe I'm missing something.
I would aim for the highest possible sample rate and then post-process the signal (assuming that's computationally reasonable, i.e. you can keep up with real time).
There are general practices/principles for minimizing noise. Having a ground plane. Separating your digital electronics from your analog electronics. Clean power supply. Shielding.
Another thing I'd be concerned about is that this whole thing is a big antenna. Yet another concern is that motion might influence the results (any time you have conductors moving in the presence of fields you get some induced current).
Commercial positioning solutions are usually based on optics and gratings. I wonder if that's a better approach even for a hobbyist. Something like an optical mouse just linearized...
I'm super impressed they built a 7 person business around this concept and have been going since 2011 (all while doing their manufacturing in Germany!)
Author here. I agree that the Rust embedded books are a nice read, and the idea of type state programming --- taking advantage of Rust's ownership and generics system to enforce at compile time transitions between logical states via "zero-sized types" --- is interesting and could be useful in some contexts.
However, that is not what is happening here.
P0 and P1 are distinct types because they are distinct hardware registers.
I think it's great that they're modeled as distinct types; the problem is simply that Rust makes it difficult to conceptually iterate over distinct types (regardless if such iteration occurs at runtime via a loop or at compile-time via an unrolled loop, as per Zig's `inline for`).
An aside about "type state programming": Microcontrollers have a lot of functionality packed into the pins (see the STM32 "Alternate Function" datasheet tables).
Trying to model all of that using ownership of zero-sized generic types would strike me as a "when all you have is a hammer"-type situation.
If a single pin switches between, for example, high-impedance, gpio low, and PWM output depending on what mode your firmware is in, I suspect it'd be a nightmare to pass owned types around Rust functions --- one would have a much easier time (and more likely to be correct) if they checked their design using something like TLA+ / Alloy or implemented the firmware using an explicit statecharts runtime like Quantum Leap's QP framework https://www.state-machine.com/.
Even if you didn't have Output Pin, couldn't you just declare a sum type?
enum MyPin { P0, P1 }
Edit: feel free to ignore, read your answer somewhere else about this
You would then have to pattern match when you read the value but I don't see a reason to reach for macros or anything more complicated.
That said, really enjoyed the read (and I'll definitely try zig at some point, if only for the speed / compile experience), even if my experience with Rust didn't match yours; my background is a bit different though, I worked with C++ and Haskell in the past, which definitely made rust feel almost natural.
Overall I'd say that the compiler helps me not to keep a lot of rust syntax in my mind and just try things until it works
>An aside about "type state programming": Microcontrollers have a lot of functionality packed into the pins (see the STM32 "Alternate Function" datasheet tables). Trying to model all of that using ownership of zero-sized generic types would strike me as a "when all you have is a hammer"-type situation.
I second this. The idea of checking that a pin is "only used in one place" doesn't really jive with how I think about microcontroller programming. It's very common for one pin to be used for multiple distinct purposes at different times.
There's also a lot of different ways of conceptually slicing pin states. For example, if you are charlieplexing LEDs than you'll switch pins between 'input' (high impedance) and 'output' modes, but at a higher level the pin is serving a single function.
> The idea of checking that a pin is "only used in one place" doesn't really jive with how I think about microcontroller programming.
The borrow checker is not checking that the pin is used in "only one place", it is checking that you don't use the same pin for two different purposes at the same time.
It make sure that you configure your pin as output pin before using it as an output pin, and that you reconfigure it to input pin when using it as such.
(And there are some escape hatch to use when the type system is not sufficient to express that different code paths are disjoint, like RefCell, with runtime check, or unsafe)
Borrow checker tracks who is using what over time. The can prevent concurrency and uncoordinated mutation, use after free type problems.
Type system checks how it is being used.
Both are tools and can used to help ensure a correct program. It really comes down to how these _tools_ are used to help the programmer and the team deconstruct and manage a system to solve a problem.
I think petgraph [1] is an excellent example of relaxing some of the constraints imposed by the tools (borrow checker, type system) to make a system that was easier to use and extend. These things are much more continuous than we realize, it isn't all or nothing.
In a lot of ways, I wish Rust's complexity was a little more gradual, or that we knew how to use it in a gradual way. Use of Rust features encourages C++ levels of complexity. Use of Zig features encourages C-ish levels of complexity.
Zig is to C
as
Rust is to C++
I also think the author had a much better model of the system and the hardware and what they wanted to accomplish during the rewrite and could better project the problem domain and the language together.
Learning Rust and the problem domain at the same time is extremely difficult and in a way leads to a perversion of both.
What do you think about modeling the hardware as a "Resource" register, port, memory, etc. Then modeling a type over a collection of resources.
The question that I would ask myself when trying to use Rust and the features it has out of the box is, "How much fine grain rigor do I want Rust to model for me?" For the keyboard scanning code, in asm or C, one might just have a function `get_keyboard_state(*keyboard_buffer)` but this exposes a sampling problem and would require the programmer to determine state transitions. So maybe a channel or an iterator would be better. Then we might nee to run it in an ISR, the hardware it uses might be multiplexed with other parts of the system, etc.
Every Rust feature needs to be weighed, or rather, given a complexity budget, every Rust feature needs to be purchased.
Zig is awesome BTW, but it doesn't make the same correctness guarantees that Rust can.
> Borrow checker tracks who is using what over time.
This is a very imprecise statement. Do you mean tracks at "compile time" or at "run time"?
A more accurate statement would be -- the borrow checker enforces a specific set of rules and constraints at _compile time_ only. But this set of constraints guarantees memory safety at run time (with the exception of unsafe code). In fact, Rust's runtime is minimal -- it handles panics but has no GC or other bells and whistles. The fancy things are in libraries.
Ah, I was going on what the OP said ("ensuring that a given pin cannot be used in multiple places").
That seems sensible, but also not particularly valuable. A lot of the time it makes sense both to 'read' and 'write' from a pin (e.g. if it's open-drain with a pullup).
> It's very common for one pin to be used for multiple distinct purposes at different times.
Anecdotal, but as someone who works in this space I haven't found this to be the case. In my experience, any particular pin is wired up for a specific purpose, and so the firmware usually just sets it to that mode as appropriate. Generally if it's found that the needed peripherals couldn't be multiplexed to pins without conflicts, it's time to move up to a package with more pins brought out.
I'm currently working on a relatively involved firmware for ATSAMD21 in Rust, and have mostly enjoyed the experience. While some of the language concepts have taken me a while to get comfortable with, and we're still figuring out parts of the ecosystem, it's quite usable and the tooling is a huge improvement over anything else I've seen.
I agree that iterating over types of a tuple is indeed not easy, but in that case, it should be trivial to iterate over an array of `&dyn OutputPin`. Why is that not working in this case?
Interesting write-up! I've barely used Rust but had/have a similar feeling. It's really more akin to C++ and really powerful but also pretty complex. For smaller MCU projects it just feels like overkill.
> Microcontrollers have a lot of functionality packed into the pins (see the STM32 "Alternate Function" datasheet tables). Trying to model all of that using ownership of zero-sized generic types would strike me as a "when all you have is a hammer"-type situation.
The whole idea of utilizing TLA+ for a system level check really does seem like something that would be awesome, even if it's unclear how much effort it'd require to instrument an entire project with TLA+.
> the problem is simply that Rust makes it difficult to conceptually iterate over distinct types (regardless if such iteration occurs at runtime via a loop or at compile-time via an unrolled loop, as per Zig's `inline for`).
Rust just brings a lot of incidental complexity along and still makes some things really difficult. Perhaps it's better in the long run but it's just harder to work with.
Similarly, I wanted a simpler language than Rust and started using Nim last summer for embedded projects. Primarily since it compiles to C which let's me target ESP32's and existing RTOS'es without rewriting everything or trying to get LLVM based tools to work with GCC ones. However, it also embraces `lazy` compilation approach to code and it's standard library.
I wanted to try your example in Nim. Here's roughly how your example would look in Nim (it appears to duck type fine as well):
var
# normally just import these from the C headers as-is
# but this lets us run it
p0* : RefPort0 = addr port0
p1* : RefPort1 = addr port1
var rows = (
( port: p1, pin: 0 ),
( port: p1, pin: 1 ),
( port: p1, pin: 2 ),
( port: p1, pin: 4 ),
)
var cols = (
(port: p0, pin: 13 ),
(port: p1, pin: 15 ),
...
(port: p0, pin: 2 )
)
proc initKeyboardGPIO() =
rows[0].port.pin[rows[0].pin].dir = output
for item in rows.fields:
item.port.pin[item.pin].dir = output
I've toyed with the thought of adding TLA+ hooks into Nim similar to Dr Nim (https://nim-lang.github.io/Nim/drnim.html) using the effect system. Not sure if Zig has an effect system for a similar setup.
Author here. Both my Zig code and the Rust peripheral access crate model the pins as distinct types, which I think is correct --- the pins have different memory addresses and sometimes (depending on the microcontroller) distinct sets of controlling registers.
The tricky part in Rust is how to make things generic across distinct types.
Zig's comptime lets you sort of "duck type" it (but with compile-time checking that all of the methods exist, etc.), whereas Rust requires that you explicitly introduce a trait and implement trait methods across the types.
The embedded HAL crates do this with extensive use of macros, for example: https://github.com/nrf-rs/nrf-hal/blob/aae17943efc24baffe30b...
This solution makes sense given the constraints of Rust, but there's quite a cost in terms of compiler time and cognitive overhead to understand what is going on.
(Aside: I didn't use the HAL in my Rust firmware, that's a higher layer of abstraction; I only used the PAC crates.)
Instances of types also have different memory addresses - that doesn't mean they're different types.
I think it's a hard problem because every microcontroller is different. A pretty common pattern in C land is to define a peripheral struct (eg I2C, GPIO) which has a 32 bit uint for each register in the peripheral, and then create a pointer to the physical memory address for each peripheral. That means you can write functions which take an I2C peripheral without knowing which one - and so if you decide later to move over to I2C2 it's just a case of changing one variable.
That works because broadly there's very little difference between I2C1 and I2C2, or GPIO0 and GPIO1, in the microcontroller. If they start being very different then you'd have problems with that approach.
Author here. I considered the enum "solution" but found the match on usize tuples to be clearer because it requires less code. Introducing an enum doesn't help because it neither:
+ helps better model the domain: P0 and P1 are already device ports, wrapping doesn't clarify anything,
+ nor does it buy you safety; arguably I'd say it makes it less safe, since the real risk with this sort of code is that you fat finger when copy/pasting between the electrical schematic and the firmware, so by adding extra wrapping you further obscure the pin assignments.
Ah, sorry I added further comment to above before seeing this. I can see why you would say that, but does the arbitrary integer risk of the 0, 1, _ match not create even greater risk? Compiler cannot help at all.
First, we should be clear that this is my goofy hobby keyboard project --- if I was concerned about safety, I'd have written it in MISRA C or something =D
There are many things about this project that a compiler can't help me with. I had to read about all of the pinouts from a PDF, draw them on a circuit board, and then map those pins in the firmware.
The code only deals with that last part, and in this particular example I decided that it was safer on the whole for the code to be obvious (easy to read + compare with hardware schematic) than to go through contortions with types to make some things more checkable by computer but less-checkable by human inspection.
The main risk here is not passing the wrong value to this match, it's fat fingering the transcription from the schematic.
Makes sense, and I guess given the above your conclusion not to use Rust seems like the correct call for your situation. Thanks for explaining further. It's true you say much of this in the article.
Yeah I think people like to match the values of a programming language to ourselves as people. (And by values I mean, correctness, speed, expressiveness, velocity, etc). Like, I’ll pick the values which I like the most and find languages which match the values I aspire towards. Eg maybe I like the idea of my programs being strongly typechecked without sacrificing speed - so I program in rust. Then I write clean rust code even when I’m doing a quick and dirty prototype.
The better & harder approach matches a language’s virtues to the problem at hand. Strict correctness doesn’t matter much for a hobby program like this - moving fast and having fun are probably more important here. Expressing that with your tools might mean using Zig, or using rust but being sloppy with allocations and .clone() because it’s fine. Or treating rust like C and using unsafe everywhere. “Making invalid states inexpressible” isn’t that important in a fun side project - unless maybe that’s fun for you!
The right question isn’t “What is my favourite tool?”. It’s “If this project had a soul, how would it want to express itself through code?”
> The right question isn’t “What is my favourite tool?”. It’s “If this project had a soul, how would it want to express itself through code?”
I somewhat disagree. What you're saying is basically "pick the right tool for the job", but more poetically (not a criticism, BTW, I like your phrasing).
But what I think this is missing is that the "right" tool is at least in part based on your favorite tools are.
Or to put it differently, using a tool that may be suboptimal for the job, but which you know extremely well, may be better than using a tool you don't know which is optimal for the job.
Of course, this is much less of an issue for hobby projects, and if one of your goals for a project is "learn new stuff" (a goal I often have for my own hobby projects), then picking the optimal language may be exactly the right choice. You get to learn a new thing while not fighting with the language.
I had a think about this and I agree. There’s some tension here - you want breadth, but you don’t have enough time to get good at every language and framework. Maybe the right approach is to have familiar ground in each domain you find yourself. Pick enough languages and frameworks so you have trails in any terrain you want to tread with your work. You don’t need to be an expert in both php and rails, in both Java and C#, or both unity and unreal. But you want enough scope that if you want to throw together a quick and dirty UI prototype, you have familiar tooling you can call your own.
For me, when I want to make a quick UI prototype I reach for JS and Svelte. Because I’m comfortable there, its not worth it to also be an expert at rails, and Php and SwiftUI and C#/WPF. But if the only tool in my toolbox was Rust, or C, or Unity or something, I’d be much worse at prototyping user interfaces. The inverse is also true - if I wanted to write a database but only knew JS, I’d be in for a rough time. You want a home base in each domain.
Oh and also, if the whole separate types for each mapping thing is so annoying and wrapping the tuples is unhelpful, which is understandable, then perhaps safer to wrap just the pins in emums and then always access them via the enum?
Then it would be (enum type, usize)
Then also at times where all Pins need handling, the.exhaustive matching can reduce risks of not doing so?
Keep in mind that this project was a lark that Nicki (http://www.nickivance.com/) and I literally made in the back of a van for her exact use case.
My article is a fun writeup of the design considerations and challenges of an email-only UI. It's not supposed to be a marketing piece, to convince you to (not) use the service, or to pass moral judgment on anyone's preferred communication medium.
I should also point out that emailing the app a Slack token is the same as emailing it a password, since the token allows it to read/write messages on your behalf (which is, of course, the whole point). Sending passwords around via email has security implications, which I'm sure you (a thoughtful, attractive, and considerate HN reader) can come to your own conclusions about based on your personal needs and risk preferences.
on the stop slacking site, I think the "captions" should appear before the images. It was initially confusing because the first caption doesn't show under 800px height screen. If I hit spacebar and scroll a page, then I see "you get mentioned on slack" and the following picture that looks like an email client, which is confusing.
Re: computing M and s, does torch.quantization.quantize_qat do this or do you do it yourself from the (presumably f32) activation scaling that torch finds?
I don't have much experience with this kind of numerical computing, so I have no intuition about how much the "quantization" of selecting M and s might impact the overall performance of the network. I.e., whether
- M and s should be trained as part of QAT (e.g., the "Learned Step Size Quantization" paper)
- it's fine to just deterministically compute M and s from the f32 activation scaling.
Also: Thanks for the tips re: CMSIS-NN, glad to know it's possible to use in a non-framework way. Any chance your example is open source somewhere?