LLM Gateway v2
Centralized gateway with cost controls, budget-aware routing, and semantic caching.
LLMOpsRoutingCost
Data Flow
1 Request 2 Tenant policy 3 Budget-aware router 4 Semantic cache 5 Providers 6 Cost attribution
Implementation Notes
Adds tenant-safe semantic caching and automated route tuning from evaluation feedback.