Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Tinygrad and rusticl and aco: why not? (airlied.blogspot.com)
46 points by pantalaimon on July 15, 2023 | hide | past | favorite | 40 comments


> large GPU stacks usually make me want to wipe the machine after I test them

Core value proposition for NixOS/GuixSD right here. That said, the amount of time you have to dedicate to figuring out how to make everything work well for your use cases might negate the benefits a bit :)


Cannot vouch for it, but the nixified.ai project is precisely geared towards this idea. Having Nix builds to run machine learning environments[0]

[0] https://nixified.ai/


Once you figure it out you never have to figure it out again.

And there is likely someone else who already figured out something as common as this so you could just import their config.


Ehh, the reality is not quite that rose-tinted; in practice you have to put the elbow grease in, still. It's normally not quite as simple as porting someones config IME, because often it's wrapped up with other things they need. I've been a NixOS user and committer for 10 years. Things do in fact change and sometimes you have to learn a new thing, often. I'm dating myself here but I still remember when the Great Haskell Package overhaul happened, in an attempt to start vendoring Haskell packages directly from upstream Hackage; before that there was totally different infrastructure and every package was added manually. That broke a lot of things.

Similarly, I only started using Flakes last year after years and years of development. Rust packaging is still pretty bad and non-granular, unless you use cargo-based generators which all have issues, and none are upstream. If you are using even more marginal languages, your experiences will probably range from bad to worse. NixOS is constant evolving and things do get deprecated and changed. You do have to learn new things.

What you don't have to deal with as often is just, things randomly breaking. And with Flakes you don't have to deal with ambient Nixpkgs changes. I have shell scripts using nix-shell I haven't had to touch in years, despite incompatible changes in the tools used by the script. I still have a walled off NixOS machine running a 15.03pre-release, IIRC. I could probably upgrade it but the software on it isn't going to break. I suspect the disk will fail first (which is fine for this machine.)

I think the best analogy is this: tools like Nix, Guix etc let you move mountains as an individual. That's impressive. It still requires you to get your hands dirty and actually push the thing, which is still work.


I love NixOS, but if you're a heavy Linux user the odds that you do something fairly unique in your personal setup gets very high. For me, to give you an example of just one little microcosm, I have probably the only combination of SwayWM, gnome-keyring and NixOS in the world that works the exact way mine does, and it took quite a while to land on it (and sometimes time to maintain it.) It's actually pretty simple and elegant now, but getting to where I am today felt anything but simple.

Back when I first started (5 years ago,) drivers for my label printer were not available, and I stumbled around trying to figure out how to make a derivation for them. Ultimately I succeeded, but for a n00b it did feel pretty ridiculously hard to try to figure it out while not even having much of a background in pure FP languages less the oddity that is Nix. I never got around to adding it to Nixpkgs, but thankfully somebody else did.

There is indeed solutions for a LOT of things. Is home manager worth the time? Not sure, I just use global configs instead of home dir configs for most things. There's multiple solutions for secrets but I have a few different Nix configuration repositories (for different sets of machines, not one for each machine; I only do this when it feels like it makes sense, like to keep servers for some org separate from my home machines) and not all of them have SOPS setup because while it works, bootstrapping it always has me guessing a little to make sure the way I'm setting it up actually makes sense.

Flakes. They're good, but I started before everyone recommended them for everything. The switching cost of flakes was not huge, but nor was it trivial, as I had to adjust a lot of workflows for it. This kind of negates the "you never have to figure it out again" thing. On the contrary, NixOS is a moving and vastly complicated machine, and it indeed feels like things need to be re-figured out quite often. Rust dependency management in Nixpkgs is sure to change in the future: right now it can't handle all of the situations that Cargo can and often requires including a copy of the lockfile in the Nixpkgs tree which seems unideal.

Worth it still? Yes, because I can throw my boot drive out the window and have a machine back up and running in like 10 minutes. All of my desktop machines are very consistent, because any configuration that matters to me goes into declarative and versioned configuration. The concept of "hosing" my OS doesn't exist anymore, and even if a workflow does break, I can trivially bisect it to figure out why. Nix tools are super complicated, but they are super powerful too. NixOS is Linux with superpowers. All of the cool stuff that open source could theoretically enable you to do, NixOS makes practical.

But is it perfect? No. To this day, you will not find a derivation for plenty of very useful closed source software. No VMWare Workstation, IDA Pro, etc. AppImage binaries that run other AppImages don't work with appimage-run; though I have a workaround with Nix-LD that does work, I've never seen anyone else doing it, so I suspect not too many people do this. Most open source software is there, but it can be a bit hit or miss at times. You'll occasionally find places where missing dependencies or hardcoded paths in the upstream sources lead to subtly broken or suboptimal things.

So if you only manage one machine with NixOS or you don't have a lot of time to invest upfront, or you have some use cases that are pathologically difficult to deal with, it's not always practical for everyone.

I think if NixOS immediately sells someone, they're in. They'll get it. If not? Maybe in a couple more years. For me, it was on first sight.


> I think if NixOS immediately sells someone, they're in. They'll get it. If not? Maybe in a couple more years. For me, it was on first sight.

I disagree.

I GET the value proposition of Nix. It's source control for your operating system and environment. This is valuable. The problem is that the activation energy for Nix is WAY higher than it ever was for source control.

Recommending Nix is more like giving a beginner the advice "You really need to learn a scripting language." I regularly get pushback from junior engineers about this. However, it is almost uniformly one of the things that sets apart the average programmer from the actively good. To be fair, I get less pushback now that Python is more common.

Nix is even worse that this. With scripting languages, you don't have to understand the universe in order to get "Baby's First" to work. With Nix, it's basically "The fields of Elysium are laid out ahead of you over there on the other side of that river. Oh, but the river is a half mile wide raging torrent of steaming sewage. But, trust us, you'll be much happier after you swim through it."

Yes, I know I will be happier. But swimming that half mile of sewage is a LOT OF PAIN (from what I can see: about 6 solid months of frustration before you "Git Gud(tm)").

That's my view of Nix.


I'm not saying that everyone who doesn't use NixOS just doesn't get it. I'm saying just about anyone who is immediately sold is probably correct.

From a romantic point of view there's obviously a lot of cool ideas in NixOS. However, it's not just that that drew me in. I suffered through trying to find good ways to manage my configurations, things like Ansible and SaltStack, and dotfiles repos, and just felt unfulfilled by any of them: they're all just not proper solutions. Kludges at best, that solve a fraction of the problem at most.

So when I saw NixOS, I got it. Not only did I get why it's good--someone who's just experienced enough would see that--but I immediately realized it solves the exact problem that I have, and that I was willing to go great lengths to get that solution.

I don't think anyone who is not a fairly experienced programmer with a lot of time and effort to burn will have a good time with Nix as it is today, though. I hope whatever comes next after NixOS answers that part, because I think everyone deserves what NixOS can make possible.


Nix gets these sorts of comments all the time, which I think is rather unfair. People are comparing Nix to things like Debian package manager and config tools, or RPM, ansible, etc. But Nix is a fundamentally different and far more powerful thing. It's like complaining that a CNC machine is so much hard to use than your swiss arm knife. They aren't the same thing. If you are a professional developer, the effort put into Nix is rewarded in kind with the fundamentally new capabilities and assurances it gives you. You mention Git being a pain in the ass to learn, but I'm sure you'd agree it behooves you to learn it and not stick to older, fundamentally different systems.


> It's like complaining that a CNC machine is so much hard to use than your swiss arm knife. They aren't the same thing.

However, 90% of my knowledge of Tormach CNC machines also applies to Haas CNC machines--there's only about 10% that is specific. Even if Tormach or Haas goes away tomorrow, most of my knowledge that I gain on one or the other is still valid.

If I learn a scripting language, 50% of it is about specifics and about 50% of it is transferrable. Learning Python gives you a lot of very similar concepts if you have to transfer to Ruby, Perl, etc.

Nix, on the other hand, is all about the specifics. If another validated, configurable, source controlled system wins over Nix, ALL of my knowledge is dead.

> If you are a professional developer, the effort put into Nix is rewarded in kind with the fundamentally new capabilities and assurances it gives you.

The idea of Nix is fine. What I question is the implementation.

We've already got a disaster of path dependence with Git. Something with a better UI should have come along (it has ... it's called Mercurial, but that's a different rant) but hasn't because of network effects.

I want the idea of Nix but I have ZERO interest in helping to lock in such a disaster of a UI.

The worst part ... I'm going to throw in. Too many people who I trust have thrown in with Nix for me to continue to sit on the fence.

But, man, I will be definitely waiting for the replacement that doesn't have such a horrific UI.


I actually don't disagree with you on the UX. It's horrible. If a group were willing to put in enough effort though, it's very possible to put together an actual GUI that edits config for you. I.e. you could have it do most of what the UIs in Ubuntu do, plus all the new features, then leave direct config editing to power users.


All fair points, i was a bit hasty with my comment.

Do you mind sharing your configs for gnome keyring with sway? I'm on sway and would love to use that.


Here's a simplified version of what I have now.

    let
      dbus-sway-environment = pkgs.writeShellScript "dbus-sway-environment.sh" ''
        ${pkgs.dbus.out}/bin/dbus-update-activation-environment --systemd DISPLAY WAYLAND_DISPLAY SWAYSOCK XDG_CURRENT_DESKTOP=sway
        ${pkgs.systemd.out}/bin/systemctl --user stop pipewire wireplumber xdg-desktop-portal xdg-desktop-portal-wlr
        ${pkgs.systemd.out}/bin/systemctl --user start pipewire wireplumber xdg-desktop-portal xdg-desktop-portal-wlr
      '';
    in
    {
        services.gnome.gnome-keyring.enable = true;
        programs.sway.extraSessionCommands = ''
          export GNOME_KEYRING_CONTROL=/run/user/$UID/keyring
          export SSH_AUTH_SOCK=/run/user/$UID/keyring/ssh
        '';
        etc."sway/config".text = ''
          # ...
          exec --no-startup-id ${dbus-sway-environment}
          exec --no-startup-id ${pkgs.dex.out}/bin/dex -a
          # ...
        '';
    }
I believe all of this is necessary. Now in my case, I have a bit more going on, but I think most of it is no longer needed. In the past it was significantly worse in my configuration :)

How this works:

1. When you set services.gnome.gnome-keyring.enable, you get a few things. It sets security.pam.services.login.enableGnomeKeyring which gives you the PAM modules needed to start gnome-keyring and pass it the password on login. It also sets up some systemd and DBus services, which are necessary for gnome-keyring to work right.

2. GNOME_KEYRING_CONTROL and SSH_AUTH_SOCK need to be set for things to work right. I traced down where they're supposed to get set in GNOME, but I don't remember why it did not work for SwayWM: I think things happened in the wrong order. So, I set them manually.

3. `dbus-sway-environment` is supposed to ensure that the DBus environment for the user session is updated with the graphical environment. I found this necessary for a number of things to work correctly. It's possible this isn't needed for gnome-keyring anymore, but it might still be. In the past I was using custom systemd user services to run the components of gnome-keyring in which case it really was needed.

4. Finally, dex. `dex -a` launches XDG Autostart applications. This is handy because when you set services.gnome.gnome-keyring.enable, several XDG Autostart applications get installed for the different modules of gnome-keyring. SwayWM itself, being a WM, does not start autostart applications, and in fact, you may not even want this behavior in general, in which case you should adjust it to explicitly start just the few keyring applications you might want.

In case you don't know, the reason why both PAM and dex start gnome-keyring is because when PAM starts gnome-keyring, it's running in a sort-of bootstrap mode: it basically just exists to store the password temporarily, then the autostart modules take over later.

addendum: I set XDG_CURRENT_DESKTOP=sway explicitly because my XDG_CURRENT_DESKTOP is normally Unity to trick Electron and other apps into using libappindicator. It may not be necessary to do this anymore, but I haven't tried to see what happens.


This is amazing, thank you for putting in the effort to document this.


I have griped about this before, but I don't really get the hype around tinygrad... GitHub is littered with ML frameworks, some of them (Like MLIR/TVM) with very good working demos right now, and other smaller scale ones (like GGML) are SOTA in very specific niches.


I don't really get the hype around tinygrad

geohot

whether that validates it or not, I think thats the reason.


It's super easy to install and use, and you can comprehend it in its entirety _because_ it's such a tiny project. Pop open the source code and browse around! The genius is the simplicity.


Calling a compiler infrastructure project like MLIR an "ML framework" feels a bit disparaging.


No, but its an increasingly popular middle layer for other frameworks.


MLIR is really cool imho


And did you gain any knowledge the last time you “griped” about it? If not then who are these “gripe” sessions even serving?


> And did you gain any knowledge

...No.

I am not against tinygrad or anything, but I can't find any projects using it as a backend. And I can't find benchmarks for, say, the stable diffusion or llama demos.

But I am frustrated that it seems to be attracting lots of attention on social media, while other ML frameworks/projects targeting wide hardware support, and already being used as backends in places, are flying under the radar.

And Hotz's very brief spat with AMD that I saw on his YouTube channel also raised my eyebrows.


What was his spat with AMD about? I think he complained about AMD's drivers being shit on the Lex Fridman podcast but that's not eye brow raising right?


Because of unfixed rocm bugs, Hotz stated that Tinygrad was going to abandon AMD support on GitHub, and was very upset about it in the following YouTube video.

This is when Lisa Su apparently stepped in and patched things up.


Maybe you should look at the code base before throwing a tantrum on a hacking blog.

Try adding a new feature to Tensorflow / PyTorch, do the same for TinyGrad and you won't have this question anymore.


Just curious, have you tried adding a feature to TinyGrad? Or are you arguing in the abstract because TinyGrad is small?

Because I personally have tried, and found the code to be so code-golf and spaghetti-like that I gave up, it just wasn’t worth my time (even for the $1000 bounty).


I did small changes, took part of the Twitch sessions and thinking and understanding it. I contributed the main ideas for Kernel search, and was happy to get George out of a mini depression with some quick fun wins. He almost gave it up, but now the progress every day is crazy.

The way to implement anything new and cool is to read it all first.

I love the way convolutions and matmuls are both expressed using ShapeTracker and simple mul operation for example. No other framework has that elegance.

Until you understand that part, it's not worth trying to change it.


Pytorch is pretty easy to add stuff to.


comma.ai uses Tinygrad as its backend


Rusticl seems a bit hidden for all the (positive) noise it creates.

I'd like to try it out, but:

- are there any distributions shipping it?

- what hardware does it support?

I can guess that distros are not shipping it due to some stability concerns (which I could not see so far). I'm also guessing that despite being a Gallium driver, some features will depend on the capabilities of the underlying hardware. Can I hope that my AMD APU will finally get some OpenCL support?


> what hardware does it support?

That’s the nice thing about it, it supports most hardware supported by Mesa going back even to r600 GPUs [0]. So anything Intel or AMD should work.

My guess for as to why distributions are not enabling it yet might be due to the Rust build dependency that might require a newer Rust version than what’s provided by the distribution.

[0] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20...


> Can I hope that my AMD APU will finally get some OpenCL support?

What APU is it? OpenCL support goes back pretty far.


I haven't found anything for the Vega 8 that would run on Linux.


? I have had OpenCL on my 4900HS for awhile, just through the Mesa OpenCL driver.

I believe my really old Llano (2011) APU even had OpenCL support. It was a selling point at the time.


Same, it just works on my Debian powered home server with 5800H with just the Mesa ICD installed.

Not that it really matters for me, I only run wgpu compute stuff over Vulkan for my experiments.


Hm, now I have to start asking questions of my OS.

For what it's worth, my Llano kept hanging up when I tried.


You are probsbly just missing the driver, some linux distros dont ship it by default.

If you are on Windows... shrug.


No mention what ACO is.

The benchmark is just a dump of the console output with no explanation what the individual phases mean.

If I am going to be honest, this is a terrible blog post.


> No mention what ACO is.

Griping about initialisms has gotta be hn's favorite pass-time by far. It's as if we don't live in a world where things can be looked up with the flick of wrist. It's as if we're not discussing this on a website entirely premised on making information access/retrieval more efficient.

> Mesa ACO compiler

More than enough disambiguation that the first hit tells me exactly what it is.

> If I am going to be honest, this is a terrible blog post.

Actually, I was wrong: cheap disparagement is hn's favorite pass-time.


AMD COmpiler


XDC 2022 | Rusticl: An OpenCL implementation written in Rust | Karol Herbst

https://www.youtube.com/watch?v=Es7UmvNAEvk




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: