I hope that person is paying you for this positive PR spin!

As others here are saying, the rules are the same for everyone in that anyone can effectively buy their freedom (even if only gradually over time). The practical difference is that most people can't afford it.

Obviously it depends on exactly what those opinions will end up being. It's hard to dispel the idea that every other big company isn't trying to find a way to do exactly the same things as the other big players, just somewhere else down the line. My anticompetitive-potential detector definitely went off as soon as I read that headline.

OpenAI function calling provides a means of producing valid JSON from a function signature, or a simplified version of a JSON schema definition; however, it has two big downsides:

1. it doesn't work with open-source models, and 2. sometimes the JSON it produces will not even parse.

A similar interface was recently merged into our open-source project Outlines. The approach can be extended to any open-source model and it can guarantee that the output will be parsable JSON.

Constructive feedback, bug reports, feature requests, and questions are greatly appreciated!

In my experience, that's always been the standard among Vim--and even some Emacs--users.

They said "terminal", not "CLI/TUI apps", so probably only using cat, sed, and ed ;)

We provide this in https://github.com/outlines-dev/outlines.

Our project Outlines provides JSON output in a near optimal way that also works for all types of pre-trained transformer-based LLMs: https://github.com/outlines-dev/outlines

Our approach also extends to EBNF grammars and LALR parsing. There's an example of that in the repository. It builds off of the Lark library, so you can use existing grammar specifications instead of starting from scratch.

Outlines looks like an actively maintained alternative to Guidance. Do you have any plans to support llama.cpp?

We've considered it, but I haven't started any work on it. Feel free to create an issue to track the status, if only to find out who else might be interested and/or working on it.

In case anyone is wondering, this is essentially a few complaints about the basic transpilation/source-to-source approach taken by Cython and then some promotion for Rust. It unfortunately mixes some general C/C++ complaints in there, too.

Beat me to it.

tl;dr "Don't use Python/Cython or C/C++. Use Rust instead, it's better." is basically that article.

We also had an implementation of grammar-driven guidance around the same time: https://github.com/normal-computing/outlines/pull/131. I imagine many others did as well, given all the papers we found on the subject. The point of this and our ongoing work is the availability of very low cost guidance, which was implemented a while ago for the regex case and expanded upon with JSON.

FYI: We've had grammar constraints available in Outlines for a while, but not using the FSM and indexing approach that makes the regex case so fast. My open PR only adds that.