China seems to be systematically working to commoditize LLMs and drive prices down as far as possible. It will be interesting to see the dynamic between that race-to-the-bottom pricing in China and US tech companies trying to extract maximum value to justify their capex-fueled growth models.
A price signal that noon is a great time to do AI inference. Unfortunately, people often want AI inference done at other times than noon, when electricity isn't negatively-priced because of abundant solar panels making it in broad sunshine.
I am sometimes wondering wouldn't the most economic thing to be to geographically distribute AI computing. Latencies can be large, but power could be cheaper if the loads are just moved around the globe as needed. This would mean that hardware sits powered off or in very low power state for larger chunk of day, but maybe that is acceptable in some scenario.
This is very likely already happening as customers of data centers doing various types of AI compute schedule their workloads to minimize their own costs.
If data center compute usage was systematically cheaper around local noon wherever in the world that data center happens to be, customers who could tolerate high latency would move their compute jobs from data center to data center as the earth spins to keep their compute happening in the cheap solar energy zone. Or maybe this is a pricing model that doesn't quite exist yet at scale, but that data center operators will introduce where and when they can as more and more solar generation for data centers comes online.
Or maybe it's not actually cost-effective because moving jobs into and out of data centers is so expensive that even the electricity/compute cost savings don't actually offset it. This is ultimately a question for the entity actually paying for the compute and responding to the price signal.
Here's what we achieved in a recent project for the automotive industry:
1. DSL with Workflow Validation: Developing a domain-specific language (DSL) empowers end-users to create and manage workflows intuitively, without needing coding expertise.
2. Dynamic Expressions & Data Flow: Incorporating runtime-evaluated expressions adds greater flexibility to workflow execution, allowing for adaptive processes.
3. Workflow Timeout & Cancellation: Implementing timeout and cancellation features ensures efficient resource utilization and quick recovery from potential issues.
4. Workflow Closure Step: Adding a workflow closure step guarantees consistent environment states across PLCs, contributing to a reliable system.
reply