LLM Routing & Orchestration
28 entriesGateways, routers, and proxy layers for managing multiple LLM providers with cost tracking, fallbacks, and load balancing.
AnythingLLM
assessA self-hosted AI chat application with workspace-isolated RAG, a zero-config desktop app, and multi-provider LLM support...
Cline
trialOpen-source VS Code extension providing an autonomous AI coding agent with a Plan/Act workflow, multi-provider LLM suppo...
Cloudflare AI Gateway
trialManaged LLM proxy on Cloudflare's edge network providing unified observability, caching, rate limiting, and multi-provid...
ComputeSDK
assessA unified TypeScript abstraction layer for executing code in sandboxed environments across multiple cloud providers via...
Dify
assessAn open-source platform for building LLM-powered applications via a visual drag-and-drop workflow builder with built-in...
each::labs
assessPre-seed AI startup providing an LLM router for 300+ models via a single OpenAI-compatible API endpoint.
Flowise
assessA lightweight visual builder for constructing AI chatbots and RAG pipelines using drag-and-drop LangChain components, de...
ForgeCode
assessOpen-source Rust-based terminal AI coding agent with three specialized built-in agents, a skills framework, ZSH plugin i...
g3
assessA Rust coding agent with modular provider abstraction, token-aware context compaction, portable Agent Skills, tree-sitte...
GoModel
assessMIT-licensed LLM gateway written in Go providing a unified OpenAI-compatible API for 10+ providers with two-layer respon...
Haystack (deepset)
trialOpen-source Python AI orchestration framework by deepset for building production-ready LLM applications, RAG pipelines,...
Hermes Agent
assessA self-improving AI agent by Nous Research that autonomously creates reusable skill files, with cross-session memory and...
Langflow
assessA visual IDE for building AI agents and RAG applications with native LangGraph integration for stateful multi-agent work...
LangGenius
assessCommercial entity behind Dify, an open-source LLM application platform with visual workflow builder and plugin marketpla...
LibreChat
trialA self-hosted AI chat platform providing a unified ChatGPT-like interface for multiple LLM providers with MCP integratio...
LiteLLM
assessAn open-source Python SDK and proxy server providing a unified OpenAI-compatible API for calling 100+ LLM providers with...
LlamaIndex
trialOpen-source MIT-licensed data framework for building RAG and document agent applications on top of LLMs, with 38k+ GitHu...
LLM Gateway Pattern
trialProxy layer between applications and LLM providers that centralizes auth, cost tracking, rate limiting, failover, and ob...
Manifest LLM Router
assessOpen-source MIT-licensed Docker-deployed LLM router for personal AI agents that uses 23-dimension keyword scoring to rou...
Nori CLI
assessOpen-source Rust-built TUI that unifies Claude, Gemini, and Codex under a single terminal interface via Zed Industries'...
Open WebUI
trialA self-hosted, provider-agnostic web interface for LLMs with built-in RAG, MCP support, RBAC, and Ollama integration for...
OpenCode
assessAn open-source AI coding agent with a terminal UI, desktop apps, and IDE extensions, connecting to 75+ LLM providers via...
OpenRouter
assessUnified API gateway providing access to 300+ LLMs from 60+ providers through a single OpenAI-compatible endpoint.
Perplexity AI
assessAI-native answer engine and agentic browser company pivoting from AI-assisted search to autonomous multi-step task execu...
Pi Coding Agent
assessA minimal, terminal-based AI coding agent with four core tools and a ~150-word prompt, supporting 20+ LLM providers and...
Portkey AI
assessEnterprise AI gateway for routing LLM requests to 250+ providers with failover, caching, guardrails, and cost management...
Thunderbolt
assessOpen-source, self-hosted enterprise AI client by MZLA Technologies (Mozilla) offering multi-platform native apps, multi-...
Vercel AI Gateway
assessVercel's unified API proxy for 100+ AI models with budget controls, automatic failover, and no token markup.
Related Reviews
The AI Engineering Stack We Built Internally — On the Platform We Ship
Ayush Thakur, Scott Roe-Meschke, Rajesh Bhatia · Apr 22, 2026
GoModel: High-Performance AI Gateway Written in Go
ENTERPILOT (organization) · Apr 22, 2026
Manifest: Open-Source LLM Router for Personal AI Agents (mnfst/manifest)
Unknown (mnfst org) · Apr 22, 2026
Thunderbolt: Mozilla's Open-Source Self-Hosted Enterprise AI Client
MZLA Technologies (Mozilla) · Apr 22, 2026
Built for Humans, Consumed by Agents: The Next Decade of Sports Digital Platforms
Mark Shannon · Apr 20, 2026