Still definitely a work in progress, I don’t have those numbers yet but will be getting them soon. It’s built on agentOS which is intended to be a way for the agents themselves to add improvements that would cut costs even more. If someone finds a new way to cut costs, the idea is that these agents find it on github or are told about it, then they implement it and start using it. That’s the self modification loop part of it. So numbers specifically are difficult to pinpoint at the moment. Cutting costs is definitely important but I wouldn’t say that’s what I’m trying to accomplish with this.
The eventual goal is a self modifying system that humans don’t have to touch, like how ants build an ant hill, no single agent has to get the whole picture. They just need to know their immediate job. Throw a project at it and let them do it to save on tokens is more of the consumer bonus, a big bonus, but a bonus nonetheless.
I’ve been making steady improvements, I’m hoping that by the end of the summer it’s much more robust than it already is.
To me, the whole point of the riddle is that it reveals the most internal bias towards either yourself or others, meaning that you do things for society or for yourself. Blues don't understand reds, reds don't understand blues. The bias is invisible to the self but it is clearly there given the huge contrast in the opinions of people.
You fail to see how anyone could choose blue, even though there are plenty of people on the internet and even in the comments here who are stating they would choose blue?
Depends on the scenario… or the number of people in the experiment. A sufficiently large number of people will guarantee votes in both bins. The specific scenario (reading this outside of a vacuum) will also have knock-on effects.
Eg: reading this into the current political landscape in the US vs reading this into another toy problem about jumping off a cliff or not will have very different outcomes and ethics.
The article makes a good point with their reframing.
"Give everyone a magic gun. They may choose to shoot themselves in the head. If more than 50% of people choose to shoot themselves, all the guns jam. The person also has the option to put the gun down and not shoot it."
The "dilemma" is asking to what lengths we should go to save people choosing to commit suicide, and does that change when they are unintentionally choosing suicide due to being "tricked" into it.
I guess that just underlines how reframing can really muddy or clarify a problem. The original problem can be mapped onto many varied scenarios with wildly different ethics.
Practically at least one person will choose blue for lulz or curiosity or as a moral compass. Shall we punish them? How does it affect survival of whole population in a long term?
"Surgical "is the kind of wordage that LLMs seem to love to output. I have had to put in my .md file the explicit statement that the word "surgical" should only be used when referring to an actual operation at the block...
you're right, they are tools. that's kind of the point. PAL is a subprocess that runs a python expression. Z3 is a constraint solver. regex is regex. calling them "surgical" is just about when they fire, not what they are. the model generates correctly 90%+ of the time. the guardrails only trigger on the 7 specific patterns we found in the tape. to be clear, the ~8.0 score is the raw model with zero augmentation. no tools, no tricks. just the naive wrapper. the guardrail projections are documented separately. all the code is in the article for anyone who wants to review it.
Selected numbers from live system runs:
Scenario , Naive shell approach , Hollow API , Savings
Code search , 21636 tokens , 987 tokens , 95%
Agent drift (cons. rate) , 35% (cold start) , 70% (with handoff) , 2x
That is a lot less than enough to justify a git clone.
reply