Figure out how a way to make this applicable to any inference server #9
Description
This would mean probably that it can work with any OpenAI-like API. We can figure this out later but we should keep this migration to using any backend in mind as we continue to develop with our own backend.