Skip to content

LLM Routing & Orchestration

28 entries

Gateways, routers, and proxy layers for managing multiple LLM providers with cost tracking, fallbacks, and load balancing.

AnythingLLM

assess
open-source

A self-hosted AI chat application with workspace-isolated RAG, a zero-config desktop app, and multi-provider LLM support...

Cline

trial
open-source

Open-source VS Code extension providing an autonomous AI coding agent with a Plan/Act workflow, multi-provider LLM suppo...

Cloudflare AI Gateway

trial
vendor

Managed LLM proxy on Cloudflare's edge network providing unified observability, caching, rate limiting, and multi-provid...

ComputeSDK

assess
open-source

A unified TypeScript abstraction layer for executing code in sandboxed environments across multiple cloud providers via...

Dify

assess
open-source

An open-source platform for building LLM-powered applications via a visual drag-and-drop workflow builder with built-in...

each::labs

assess
vendor

Pre-seed AI startup providing an LLM router for 300+ models via a single OpenAI-compatible API endpoint.

Flowise

assess
open-source

A lightweight visual builder for constructing AI chatbots and RAG pipelines using drag-and-drop LangChain components, de...

ForgeCode

assess
open-source

Open-source Rust-based terminal AI coding agent with three specialized built-in agents, a skills framework, ZSH plugin i...

g3

assess
open-source

A Rust coding agent with modular provider abstraction, token-aware context compaction, portable Agent Skills, tree-sitte...

GoModel

assess
open-source

MIT-licensed LLM gateway written in Go providing a unified OpenAI-compatible API for 10+ providers with two-layer respon...

Haystack (deepset)

trial
open-source

Open-source Python AI orchestration framework by deepset for building production-ready LLM applications, RAG pipelines,...

Hermes Agent

assess
open-source

A self-improving AI agent by Nous Research that autonomously creates reusable skill files, with cross-session memory and...

Langflow

assess
open-source

A visual IDE for building AI agents and RAG applications with native LangGraph integration for stateful multi-agent work...

LangGenius

assess
vendor

Commercial entity behind Dify, an open-source LLM application platform with visual workflow builder and plugin marketpla...

LibreChat

trial
open-source

A self-hosted AI chat platform providing a unified ChatGPT-like interface for multiple LLM providers with MCP integratio...

LiteLLM

assess
open-source

An open-source Python SDK and proxy server providing a unified OpenAI-compatible API for calling 100+ LLM providers with...

LlamaIndex

trial
open-source

Open-source MIT-licensed data framework for building RAG and document agent applications on top of LLMs, with 38k+ GitHu...

LLM Gateway Pattern

trial
pattern

Proxy layer between applications and LLM providers that centralizes auth, cost tracking, rate limiting, failover, and ob...

Manifest LLM Router

assess
open-source

Open-source MIT-licensed Docker-deployed LLM router for personal AI agents that uses 23-dimension keyword scoring to rou...

Nori CLI

assess
open-source

Open-source Rust-built TUI that unifies Claude, Gemini, and Codex under a single terminal interface via Zed Industries'...

Open WebUI

trial
open-source

A self-hosted, provider-agnostic web interface for LLMs with built-in RAG, MCP support, RBAC, and Ollama integration for...

OpenCode

assess
open-source

An open-source AI coding agent with a terminal UI, desktop apps, and IDE extensions, connecting to 75+ LLM providers via...

OpenRouter

assess
vendor

Unified API gateway providing access to 300+ LLMs from 60+ providers through a single OpenAI-compatible endpoint.

Perplexity AI

assess
vendor

AI-native answer engine and agentic browser company pivoting from AI-assisted search to autonomous multi-step task execu...

Pi Coding Agent

assess
open-source

A minimal, terminal-based AI coding agent with four core tools and a ~150-word prompt, supporting 20+ LLM providers and...

Portkey AI

assess
vendor

Enterprise AI gateway for routing LLM requests to 250+ providers with failover, caching, guardrails, and cost management...

Thunderbolt

assess
open-source

Open-source, self-hosted enterprise AI client by MZLA Technologies (Mozilla) offering multi-platform native apps, multi-...

Vercel AI Gateway

assess
vendor

Vercel's unified API proxy for 100+ AI models with budget controls, automatic failover, and no token markup.

Related Reviews

Related Topics