Viruses are just another "mutagen". No different from UV causing DNA damage in your skin cells, other than the mechanism in which it occurs. The cause for cancer is well-known and, in hindisght, obvious, which is mutation.

The challenge though is mutations can happen in a plethora of ways and their effect is highly dependent on which gene is mutated. There is also the tissue context, e.g. inflammation, spatial structure, etc., that can setup a background for increased mutation. That is why targeted therapies are often the most effective, because they target the general causative feature of a given tumour subtype, the problem is not every protein can be targeted now and each tumour, even within the same subtype has their own unique mutational profile due to the stochasticity of the way mutations occur over repeated rounds of cell division.

And back to viruses, yes they cause cancer because they can mutate DNA. But it's pretty clear, most of the viral "enriched" cancer types are generally in places where transmission is commonplace, e.g. reproductive organs or head/neck.

If you think these types of tools are going to be generating "the most and best research coming out of any lab", then I have to assume you aren't actively doing any sort of research.

LLMs are undeniably great for interactive discussion with content IF you actually are up-to-date with the historical context of a field, the current "state-of-the-art", and have, at least, a subjective opinion on the likely trajectories for future experimentation and innovation.

But, agents, at best, will just regurgitate ideas and experiments that have already been performed (by sampling from a model trained on most existing research literature), and, at worst, inundate the literature with slop that lacks relevant context, and, as a negative to LLMs, pollute future training data. As of now, I am leaning towards "worst" case.

And, just to help with the facts, your last comment is unfortunately quite inaccurate. Science is one of the best government investments. For every $1.00 dollar given to the NIH in the US, $2.56 of economic activity is estimated to be generated. Plus, science isn't merely a public venture. The large tech labs have huge R&D because the output from research can lead to exponential returns on investment.

" then I have to assume you aren't actively doing any sort of research."

I would wager hes not - he seems to post with a lot of bluster and links to some paper he wrote (that nobody cares about).

What tools are "actually working" as of a few years ago? Foundation models, LLMs, computer vision models? Lab automation software and hardware?

If you look at the recent research on ML/AI applications in biology, the majority of work has, for the most part, not provided any tangible benefit for improving the drug discovery pipeline (e.g. clinical trial efficiency, drugs with low ADR/high efficacy).

The only areas showing real benefit have been off-the-shelf LLMs for streamlining informatic work, and protein folding/binding research. But protein structure work is arguably a tiny fraction of the overall cost of bringing a drug to market, and the space is massively oversaturated right now with dozens of startups chasing the same solved problem post-AlphaFold.

Meanwhile, the actual bottlenecks—predicting in vivo efficacy, understanding complex disease mechanisms, navigating clinical trials—remain basically untouched by current ML approaches. The capital seems to be flowing to technically tractable problems rather than commercially important ones.

Maybe you can elaborate on what you're seeing? But from where I'm sitting, most VCs funding bio startups seem to be extrapolating from AI success in other domains without understanding where the real value creation opportunities are in drug discovery and development.

These days it's almost trivial to design a binder against a target of interest with computation alone (tools like boltzgen, many others). While that's not the main bottleneck to drug development (imo you are correct about the main bottlenecks), it's still a huge change from the state of technology even 1 or 2 years ago, where finding that same binder could take months or years, and generally with a lot more resources thrown at the problem. These kinds of computational tools only started working really well quite recently (e.g., high enough hit rates for small scale screening where you just order a few designs, good Kd, target specificity out of the box).

So both things can be true: the more important bottlenecks remain, but progress on discovery work has been very exciting.

As noted, I agree on the great strides made in the protein space. However, the over saturation and redundancy in tools and products in this space should make it pretty obvious that selling API calls and compute time for protein binding, annd related tasks, isn’t a viable business beyond the short term.

I don’t think the basic usage of references is hard to grok for a beginner. If you aren’t going to mutate data and only access it, then pass a reference. No need for over-complicated semantics when describing it to a new Rust user.

What if you want to add a reference field to a struct? That's the point where I usually get pretty upset with how Rust works.

You probably don't want a reference field in a struct, at least to begin with. It's a lot easier to reason about structs if they contain owned data only, and you take a reference to the entire struct. There are some specific cases where it might be sensible, or even necessary to do that, but for someone who is still learning lifetimes, these cases are unlikely to come up.

I recommend reading Steve Klabnik's "When should I use String vs &Str?" post, which is generally good advice when deciding between owned data and references. The "Level 4" section covers the case of references in structs.

You get a lot more compiler help than when you try and put a reference in a class in C++, and if you want to use smart pointers its even better because you'll never have to learn about move and copy constructors or copy and move assignment operators.

What's your opinion on the veracity of this benchmark - given o3 was fine-tuned and others were not? Can you give more details on how much data was used to fine-tune o3? It's hard to put this into perspective given this confounder.

I can’t provide more information than is currently public, but from the ARC post you’ll note that we trained on about 75% of the train set (which contains 400 examples total); which is within the ARC rules, and evaluated on the semiprivate set.

That's completely understandable - leveraging the train set. But what I was trying to say is that the comparison is relative to models that were actually zero-shot and not tuned. It isn't apples to apples, it's apples to orchards.

Somewhat related — I’d love to hear your thoughts on dex-Lang and Haskell for array programming?

I still am excited by Dex (https://github.com/google-research/dex-lang/) and still write code in it! I have a bunch of demos and fixes written, and am just waiting for Dougal to finish his latest re-write before I can merge them.

Ahh that's an interesting paper I must of missed that one - thanks for the link. I think another paper that recently got a lot of hype has been the Matroyshka representation learning paper -- essentially training models with different parameters and output embedding sizes at the same time, basically distillation during training rather than post-training (https://arxiv.org/abs/2205.13147).

Afaik, they aren't really trained independently -- for most models, e.g. DINO, etc., the smaller sizes are actually distilled from larger models. It's much easier to generate performant models at smaller size via distillation.

And I'd be curious of the utility of model that scales up and down at inference - if this was the case you'd still need to have storage that is the same as the maximum model size. This would essentially be useless for embedded applications, etc., unless you have heavy quantization - but quantization in a small parameter space would probably make the smaller modes useless. I could see the benefit here in terms of optimizing latency for different applications but maybe you have other ideas.

Given all that, I think training for smaller number of parameters, as noted in OP, would kind of beat out some model that scales at inference time - especially when most people know what kind of application they are aiming to build and the required level of performance.