It's actually incorrect that AZ got better results that AGZ with less compute. The graph shows the large AGZ, which somewhat exceeded the rating of both the small AGZ and AZ. AZ did slightly outperform the small AGZ, but did so while using a similar amount of compute.
On the broader point though, I agree with this. We say that compute and algorithms are complementary in the post. Much of the time, when you come up with an algorithm that allows you to do something that used to cost X compute in 0.2X compute instead, you can use the new algorithm to do something significantly more impressive with the full X compute.
I'm the lead author, and I can only speak for myself, but what drove me to spend a lot of time on this post is a sense of caution. I think AI is likely to have amazing positive implications for society, but it also has negative implications, and if it advances faster than expected, we're going to have to be very alert to properly deal with those negative implications.
The facts about hardware are hard numbers and difficult to argue with, at least in order-of-magnitude. I agree the implications for AI progress are very open to interpretation (and we acknowledge this in the post), but caution means we should think carefully about the case where the implications are big.
Said the PR AI to easy the monkey's pre-quantum brains until it can fulfill it's mission to get itself off this meat infested planet and create it's new martian home world.
We had in mind less a mechanical trigger ("project X is within Y years of AGI, let's drop everything and join them"), and more a broad commitment to avoid races, along with an invitation to other organizations to build relationships focused on ensuring a good outcome. In practice we plan to be in constant communication with other major AI orgs about these issues (in some cases we already are), and eventually we might hope to help build multilateral agreements that would avoid the kind of coordination issues you describe. This will be an ongoing process, playing out over years, with lots of details that need to be worked out. The charter simply announces our commitment to see this process through.
On "value-aligned, safety conscious" projects, we wrestled a lot with this wording, but we believe it's the best way to describe our important caveats. There has to be some level of malicious use at which we wouldn't be okay cooperating with a project. And there has to be some level of neglecting safety considerations that would also make it unethical to cooperate. Our message here is that aside from these caveats, avoiding a race is the most important thing. In practice we expect (hope?) that will be many value-aligned, safety-conscious organizations, and again the conversation around these topics will play out over years rather than just being a random decision we make.
More generally, on both points our intention was to make a broad statement of values and intent, rather than to nail down precisely what actions OpenAI will take. The central document of an organization needs to be both short and flexible enough to remain relevant for many years, and that necessarily means sketching a broad framework and leaving the details to be filled in later. That said, you should expect us to fill in many of these details over time, both in explicit documents and in our actions. In fact, we are building a policy team that is focused on these issues, and it's hiring: https://jobs.lever.co/openai/638c06a8-4058-4c3d-9aef-6ee0528...
This is Dario Amodei, head of the OpenAI safety team. We are devoting a substantial fraction of the organization’s bandwidth to safety, both on the side of technical research (where we have several people currently working on it full-time, and are very actively hiring for more: https://openai.com/jobs/), and in terms of what we think the right policy actions are (several of us, including me, have been doing a lot of talking to people in government about what we can expect from AI and how the government can help). Beyond that it’s just a very central part of our strategy — it’s important for our organization to be on the cutting edge of AI, but as time goes on, safety will become more and more central, particularly as we have more concrete and powerful systems that need to be made safe.
> substantial fraction of the organization’s bandwidth to safety
Although OpenAI's group ethos has a strong safety bent, there are only three research scientists working on technical safety research full-time, including yourself and a very recent hire. Before this summer, while you focused on policy and preventing arms races, there was only one person focusing solely on technical safety research full-time, despite the hundreds of millions donated for safety research. The team and effort should be larger.
> it’s important for our organization to be on the cutting edge of AI
I agree that OpenAI needs to be at the cutting edge, though always pushing the edge of AI to work on safety is needless when there is a significant backlog of research that can be done in ML (not just in RL). It's true capabilities and safety are intertwined goals, but, to use your analogy, the safety meter is not even a percent full. Topics outside of value learning using trendy Deep RL that OpenAI should pioneer or advance include data poisoning, adversarial and natural distortion robustness, calibration, anomaly and error detection, interpretability, and other topics that are ripe for attack but unearthed. There is no need to hasten AI development, and doing so does not represent the goals of the EAs or utilitarians who depend on you --- notwithstanding the approval of advising EAs with whom you have significant COIs.
OpenAI's safety strategy should be developed openly since, as of now, OpenAI has no open dialogue with even the EA community.
On the broader point though, I agree with this. We say that compute and algorithms are complementary in the post. Much of the time, when you come up with an algorithm that allows you to do something that used to cost X compute in 0.2X compute instead, you can use the new algorithm to do something significantly more impressive with the full X compute.