The OpenZoo API is the easiest way to use open-source LLMs because:
- You already know how to use it: simply use the official OpenAI client for your language of choice by changing the base_url parameter
- You don't need to select a model from the ever-changing leaderboards - just provide a spec like 'chat', 'summarization', 'code' or 'math', we always serve the latest best model for the job
- We keep adding features so you can write less code: If you specify 'chat safe' (releasing next Friday!), we add a Llama-guard circuit-breaker to keep your responses safe! Soon, you will be able to just specify 'chat safe eu-compliant spanish' and save precious lines of code that you will need to maintain.
- When you need to, you can still specify exactly which model you want: simply specify the HuggingFace handle like 'mistralai/Mixtral-8x7B-Instruct-v0.1' and all parameters supported by the Openai client to get full control when you need it.
We are in closed alpha right now, watch this space!