SIE
Self-hosted inference server running many open models behind one API
What it is
SIE is an inference engine that serves over a hundred open models from a single cluster, covering embeddings and reranking, OCR to markdown, entity extraction, classification, content safety, translation, vision, transcription, and LLM tool calling. Models load on demand with LRU eviction, and the API is OpenAI-compatible for embeddings and chat completions. Teams building agents and retrieval pipelines run it on their own GPUs or on Kubernetes using the bundled Helm charts, autoscaling, and Grafana dashboards, rather than standing up one model server for each job.
