from https://voxleone.github.io/FunctionalUniverse/pages/executiv..., "The Functional Universe models reality as a history built from irreversible transitions, with time emerging from the accumulation of causal commitments rather than flowing as a primitive parameter." Is it fair to say that time is simply a way of organizing a log file on a dynamic reality? I interpreted "composition of transitions" as a system of processes. I think the hard modeling problem is interpreting interactions between processes - that transitions don't simply compose, that observed transitions may be confounded views of more complex transitions. I gather NCA would be granular enough to overcome that.
That’s a very good objection, and it’s pointing at a real pressure point in our framework.
Short answer: it’s close, but incomplete. It’s not that time organizes a log of reality; rather, reality is the accumulation of committed transitions. What you’re calling a ‘log’ it’s the ontological structure itself.
I gather you're basically saying: what we see as a transition ≠ what’s actually happening at the fundamental level. This is a legitimate and deep problem.
You’re right that observed transitions may not compose cleanly. In the Functional Universe, composition is a property of fundamental transitions. What we observe are often coarse-grained projections of many underlying transitions, which can obscure compositional structure.
The Percepta stuff would seem to demonstrate a mechanism for implementing "thinking". I don't understand how foundation models implement "thinking", but my intuition is that models are specifically trained for matching on and following procedural patterns. A task in a given domain can be performed through an associated and encoded procedure. The model holds all the linkages, as weights, that allows a procedure to be conditionally incrementally generated and performed. Does anyone have any insights about how LLM "thinking" is trained and coded?
Basically just madlibs - the models generate intermediate tokens that help predict a better answer based on training (RLHF & otherwise). They tend to look like "reasoning" because those tokens correlated with accepted answers during training.
Extended thinking passes are just more of the same. The entire methodology exists merely to provide additional context for the autoregression process. There is no traditional computation occurring
This work was performed by people across 13 institutions, invited and coordinated through the team at Northeastern. A research "swarm" seems like a great model for this kind of work. I'm curious about how it was funded, I didn't see any acknowledgements that way. The intro references the NIST Agent Standards Initiative. Also, the acknowledgement to "Andy Ardity" should for "Andy Arditi"?
Well - I've learned what a "formant" is today. Looking at the repo, it's not obvious to me what .md is authored by you vs system-generated. This is an observation, not a criticism. I was looking for the prompts you used to specify the papers summarization, which is very nice.
"We consider composite media with a broad range of scales, whose
effective properties are important in materials science, biophysics, and
climate modeling. Examples include random resistor networks, polycrystalline media, porous bone, the brine microstructure of sea ice, ocean eddies, melt ponds on the surface of Arctic sea ice, and the polar ice packs themselves. The analytic continuation method provides Stieltjes integral representations for the bulk transport coefficients of such systems, involving spectral measures of self-adjoint random operators which depend only on the composite geometry. On finite bond lattices or discretizations of continuum systems, these random operators are represented by random matrices and the spectral measures are given explicitly in terms of their eigenvalues and eigenvectors. In this lecture we will discuss various implications and applications of these integral representations. We will also discuss computations of the spectral measures of the operators, as well as statistical measures of their eigenvalues. For example, the effective behavior of composite materials often exhibits large changes associated with transitions in the connectedness or percolation properties of a particular phase. We demonstrate that an onset of connectedness gives rise to striking transitional behavior in the short and long range correlations in the eigenvalues of the associated random matrix. This, in turn, gives rise to transitional behavior in the spectral measures, leading to observed critical behavior in the effective transport properties of the media."
A big assumption with this change is that the "Modular Open Systems Approach" (MOSA) [0] [1] will be adequate for integrating new systems developed and acquired under this "fast track". MOSA appears to be about 6 years old as a mandate [2] and is something that big contractors - SAIC, BAI, Palantir [3] - talk about. But, 6 years seems brand new in this sector. I'd be curious to see if LLM's have leverage for MOSA software system integrations.
This was a good read. I was struck by the quantity of nuanced and applied knowhow it took to build SmolLM3. I am curious about the rough cost it took to engineer and train SmolLM3 - at ~400 GPUS for a least a month, and, based on the set of book co-authors, 12 engineers for at least three months. Is $3-5M a fair ballpark number? The complement is how much experience, on average, the team members had doing ML and LLM training at scale before SmolLM3. The book is "up" on recent research, so I am surmising a phd-centric team each with multiple systems built. This is not commodity skill. What the book suggests to me is that an LLM applications start up would best focus on understanding the scope and knowhow for starting from post-training.
reply