What is AI Gateway?
AI Gateway is an enterprise-grade gateway for managing AI models, tools, and agents, powered by Azure API Management. It provides security, performance, observability, cost control, and extensibility (including MCP protocol support) for production AI applications. The repository offers 30+ hands-on Jupyter notebook labs, Bicep infrastructure templates, and APIM policies.
How to use AI Gateway?
Clone the repository, install Python 3.12+ and uv, open the notebooks in VS Code with the Jupyter extension, and follow step-by-step instructions. Each lab deploys to your Azure subscription using provided Bicep templates and APIM policies.
Key features of AI Gateway
- 🔐 Security via OAuth 2.0, managed identities, and content safety filtering
- ⚡ Performance with load balancing, semantic caching, and request routing
- 📊 Observability through token metrics, built-in logging, and tracing
- 💰 Cost control using rate limiting, quota management, and FinOps framework
- 🔌 Extensibility with MCP protocol support, function calling, and multi-model routing
- 🏛️ Labs follow Azure Well-Architected Framework pillars (security, reliability, performance, operations, cost)
Use cases of AI Gateway
- Manage and control access to Large Language Models with enterprise-grade policies
- Enable secure tool access using MCP protocol and function calling capabilities
- Build and control agentic applications with orchestration frameworks
- Implement token rate limiting, semantic caching, and backend pool load balancing
- Automate failover between AI models using circuit breaker and retry patterns
コメント