normal people talk and write with some notion of meter, the cadence of communicating where pauses are inserted at places that naturally suit the speaker (and listener) to pause for thought. LLM's don't really do that, they just write a bunch of sentences.
> Researchers have found that some neurons inside the FFN are strongly associated with specific concepts or facts. One neuron might activate strongly on Eiffel-Tower-related text. Another on programming languages. Another on past-tense verbs.
People don't really write like this and they don't really talk like this (and no, people don't necessarily write exactly how they talk because they don't read exactly how they listen; the written word can be backtracked while the heard cannot, and speakers/writers know this, either consciously or unconsciously). A person would probably structure this more like:
> Researchers have found that some neurons inside the FFN are strongly associated with specific concepts or facts. For example, there could be one neuron that activates strongly on Eiffel-Tower-related text, another that activates strongly on programming languages, a third neuron activating on past-tense verbs, and so on.
Usually people wouldn't write "Another on programming languages." as a standalone sentence like that because the periods introduce an unnatural pause like they're giving a TED talk, unless of course they were punctuating that way for effect, but you'd essentially never communicate with that effect full time.
I don’t disagree with your conclusion that this is likely ai rewritten, but I do find it strange that you say “normal people don’t write like this” when it is mimicking how people write, and using patterns I have seen people write. I think models are at the point where style is not really reliable as an indicator anymore.
people sure do write like that, in novels. nobody writes scientific articles like novels, because scientific articles don't need to maximally capture audience attention. the purpose of a scientific article is to convey information - this pursuit is not assisted by punchy prose.
A lot of the common patterns people ping as AI (like "it's not X, it's Y"*) are marketing-speak, of which there's a lot of on the internet. It's applying existing patterns in unusual locations, ignoring the original context.
The one they're pointing out (the short punchy sentences) also apply to things like politicians and news articles. Blog posts are a bit odd.
* And here I mean those literal exact words. People are also extrapolating to similar patterns that use different or more words than "it's not" and "it's", but those flow better and aren't what I'm referring to here.
I'm sure there's plenty of writing in the above style to be found on the Internet, and hence having been trained on by the LLM. I'm also not a fan of this style, and in particular I'd say it's rarely or never found in scientific / technical writing meant to convey understanding rather than sell or hype. So here it's IMO more of a style mismatch.
No, but sufficiently-advanced overfitting would lead to to the model keeping track of an author stylistic profile, in the same way it keeps track of the plot of a story it's writing (i.e., badly, but well enough that you have to pay attention to notice that something is wrong).
It is trained on its own slop. They haven’t trained these models on books for three years at this point. Only on generated slop. (And RL slop upvotes/downvotes from users)
in Chicago near Wrigley Field (the Cubs stadium), the closest train stop (Addison) was basically wall to wall (some on the floor too) DraftKings advertisements until recently because they have a physical sports betting bar adjacent to Wrigley Field. After the latest round of elections they're closing that location so the ads came down.
Or subtly misrepresent politically inconvenient facts, or gently steer you into opinions based on a synthesis of broker data and demographic info, or quietly flag you in some database column due to exhibiting dissident-adjacent ideas or behaviors, or...
Yeah, they probably aren't doing (most of) these now, but it doesn't take much mental energy to extrapolate once you factor nearly every other tech company's ethical trajectory and the current geopolitical environment. Substituting classic search entirely with LLMs is not a savvy move.
I remember a few years ago memes were going around about how ChatGPT responded differently to "do Israelis deserve human rights?" ("Of course! Everyone deserves human rights...") and "do Palestinians deserve human rights?" ("While everyone deserves human rights, it's complicated... ")
Certainly, but with (what I consider to be) a key distinction: classic search, by definition, must serve information from many distinct sources outside the control of the search company.
A search engine could certainly tamper with which of these sources they surface/rank higher (which I suspect happening more often of late), but they're still obliged by their nature to branch out and seek information from the broader world.
LLMs, on the other hand, are self-contained opaque monoliths that can be conditioned to deceive or obfuscate with devious cleverness, and all control over their behaviors is entirely concentrated in the hands of whatever corporation trains them.
My thought here is that there are many. They have proven to be commodities in most use cases.
As soon as one gets annoying, expensive, advertiser heavy etc. you just rip it out and replace it with the other one. AFAICT there is zero lock-in or moat. I often am able to switch models in one click or command. This is why all the LLM providers are desperate for a product layer/comprehensive tool set.
Sure maybe they all end up that way, but there’s plenty of reasons corporate customers will want private LLM usage that is not skewed towards advertising. I am happy to pay for that.
Also, open source models are a bulwark against another search style ad Monopoly.
oh i’ve definitely seen “we’re going to track the number of bugs created in jira per team” turn into “people just file things as tasks instead of bugs” or “only easy things are filed as bugs and completed right away”. It’s trivially gameable.
working in a large codebase I use Claude for code understanding and the code reviews from Macroscope have caught bugs for me a bunch of times. Usually if I use claude it’s for refactoring a and source to source transformations that would be too confusing for me to figure out how to do with e.g. ast-grep, but that I can prompt in a minute or two and then have claude work through it. It’s stuff I could do without LLMs but it’s less effort to use them. I don’t let it write new code, because it decays the process of programming as theory building.
the last time I went to Japan was I think 2015 and the exchange rate was about 120 yen to the dollar. I bought almost all of the clothing that I wore for the next year or two during a stretch of three days in Tokyo. The exchange rate right now is 155 yen to the dollar and prices on everything in the US have gone through the roof, so this doesn't seem all that ridiculous to me. I am more annoyed by the assumption that I live in SF than the idea that I might go from SF to Tokyo on a vintage shopping trip.
As much as I love my Vita having access to Chinese handhelds with decent screens that can emulate almost everything under the sun (including PC, Switch and some Vita!) is pretty damn awesome!
> Researchers have found that some neurons inside the FFN are strongly associated with specific concepts or facts. One neuron might activate strongly on Eiffel-Tower-related text. Another on programming languages. Another on past-tense verbs.
People don't really write like this and they don't really talk like this (and no, people don't necessarily write exactly how they talk because they don't read exactly how they listen; the written word can be backtracked while the heard cannot, and speakers/writers know this, either consciously or unconsciously). A person would probably structure this more like:
> Researchers have found that some neurons inside the FFN are strongly associated with specific concepts or facts. For example, there could be one neuron that activates strongly on Eiffel-Tower-related text, another that activates strongly on programming languages, a third neuron activating on past-tense verbs, and so on.
Usually people wouldn't write "Another on programming languages." as a standalone sentence like that because the periods introduce an unnatural pause like they're giving a TED talk, unless of course they were punctuating that way for effect, but you'd essentially never communicate with that effect full time.
reply