Outside of pure security, we've seen many choose Onyx for flexibility + connectedness.
One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise).
Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example).
Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors.
The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly.
Is running your llm through azure insecure? I mean more so than running anything on cloud? My understanding was that azure gpt instances were completely independent with the same security protocols as databases, vms, etc.
Azure wouldn't be if you have your company AD/Oauth, I'm GUESSING running local models with data transfer might expose that communication if your local machine is compromised, or someone else's, potentially is multiple points of leakage, companies generally like to limit that risk. This is all an assumption btw.
Ah I see.. That makes a bit more sense and definitely adds a value multiplier for enterprises I would imagine! I'll try out the open source one and see how it works out!
That's a fair point! We were aware of onnx, but felt it was okay since they are very different products so we felt that there wouldn't be too much confusion (people generally know which onyx/onnx they are looking for).
You're almost literally in the same ecosystem, it's not like one is a Chat UI for LLMs and the other a super market, but a ecosystem of open source machine learning software, libraries and tools. That the pronunciation is identical makes it untenable, you really need to reconsider the name, discussions in person will get confusing.
We're definitely looking to add back some of that flexibility / customizability. I don't think you have to sacrifice a nice, simple UI to provide what power users are looking for.
For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice.
There are also many teams we work with that want to (1) retain model flexibility and (2) give everyone at the company the best model for the job. Every week? a model from a different provider comes out that is better at some tasks than anyone else. It's not great to be locked out from using that model since you're a "ChatGPT" company.
Yea, the license is modeled after the Gitlab license. All of the core chat/RAG/agent logic is fully MIT, and >99% of deployments of Onyx are using the "community edition"!
You don't need any subscription to run the code! By default, none of the enterprise code runs (and it can all be completely removed and the app will work as expected). Fully FOSS version here: https://github.com/onyx-dot-app/onyx-foss.
As I see it has whitelisting and enterprise integrations.. as for the OS version maybe you need to roll your own. This is a usual monetization method though.
Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows.
Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.
Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.
Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).
Agree that's a lot of other projects out there, but why do you say the Vercel option is more advanced/mature?
The common trend we've seen is that most of these other projects are okay for a true "just send messages to an AI and get responses" use case, but for most things beyond that they fall short / there a lot of paper cuts.
For an individual, this might show up when they try more complex tasks that require multiple tool calls in sequence or when they have a research task to accomplish. For an org, this might show up when trying to manage access to assistants / tools / connected sources.
Our goal is to make sure Onyx is the most advanced and mature option out there. I think we've accomplished that, so if there's anything missing I'd love to hear about it.
Alright let's say im tasked with building a fancy AI-powered research assistant and I need onyx or Vercel's ai-chatbot sdk. Why would I reach for onyx?
I have used vercel for several projects and I'm not tied to it, but would like to understand how onyx is comparable.
Benefits for my use cases for using vercel have been ease of installation, streaming support, model agnosticity, chat persistence and blob support. I definitely don't like the vendor lock in, though.
Not wanting to use Vercel is honestly a good enough reason. If you’re a heavy Vercel user you probably aren’t their target market since they’re aiming at enterprise types from what it looks like.
One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise).
Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example).
Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors.