Solar panel breakage doesn't depend on a graphics card.

Solar power is not free.

You can decide to not fix the panel that was being used to power a GPU.

Or you could sell the power back.

Or you could put it in a battery bank for when the sun is down.

Or, if none of those are the case and you just have excess power that's useless for anything but a GPU, then you prepaid for the GPU.

I love that we have solar panels, but we weren't gifted them. Using power has a cost.

It's actually common to be in that situation where the grid is paying pennies on the dollar and you have extra generation. Most grid-tie systems are in that boat.

Suddenly you find yourself looking for something to spend power on so it doesn't go to waste.

ollama is a wrapper on top of llama.cpp, and it makes llama.cpp slower, why use it?

Also Ollama has other issues (like forgetting what it really is - a wrapper).

Why would anyone use Ollama at all (aside from obvious reasons one can look up online) - llama.cpp used directly, without this wrapper is faster.

Basically one has two real choices for local LLMs: llama.cpp (if single user) or vLLM (if multi-user/enterprise).

Is autocomplete using LLMs really useful? Even with frontier models I found it to be about 50% right, I turned it of and prefer to use IntelliJ built-in, it is way more reliable.

For me local models is all about quality, and how to achieve that - e.g. by providing guardrails that test the job done.

People are using 3090 (24GB) to run models, and it is the most cost effective way to run the. Yes, it is 2x faster, but memory wise you surely can spend 24gb on llm.

Also there are smaller, still usefull models that can run on 8GB or less.

Ollama is not a good choice - https://sleepingrobots.com/dreams/stop-using-ollama/

As for oprncode, doesn't the system prompt eat too much of the context? Local models are really constraint in regards contex, and opencode AFAIR uses a 10k of it or some thing close.

Yeah, this is quite concerning.

And FedRamp has some issues with data being sent out.

Our corp doesn't allow usage of local models because of concern about potential "agent sends out code to the net" issues.

> As long as the company states the time period

But they don't, they have the "30 days", but just after that they add "unless ....". So the time period is vague.

> how long they keep it

AFAIR it is not clear, because they write it is "30 days, but ...":

> After 30 days, the data is deleted automatically, except in the rare cases where it's part of a safety investigation or we're legally required to keep it.

So you have a vague clause saying "when" and vague clause saying for "how long". If it will fly I would be surprised.

This is all pretty standard with GDPR.

> Note that Anthropic has committed not to train models on logged data, so I don’t understand some of the concerns here. What exactly is your threat model? That Anthropic would train models contrary to their terms of service? That you trust them enough not to log your data prior to this, but not enough to trust their stated limits on how logged data will be used now?

It is a different thing when they say they don't store your data.

And when they say they store your data for 30 days and review it for "issues", it makes your "spider sense" tingle. Who and how will review it, what are the "issues" they are looking for, etc. It is to vague and they can keep it this "dangerous" model for themselves.