Awesome Production Generative Artificial Intelligence

This repository contains a curated list of awesome open source libraries that will help you deploy, monitor, version, scale, and secure your production generative artificial intelligence systems and applications 🚀

You can keep up to date by watching this github repo to get a summary of the new production ML libraries added every month via releases 🤩

Additionally, we provide a search toolkit that helps you quickly navigate through the toolchain.

Quick links to sections on this page

🤖 Agent Development	☎️ Context Management	🎞️ Memory Management	🚀 Model Deployment
🎮 Prompt Engineering	🎨 Visual Engine

Contributing to the list

Please review our CONTRIBUTING.md requirements when submitting a PR to help us keep the list clean and up-to-date - thank you to the community for supporting its steady growth 🚀

Want to receive recurrent updates on this repo and other advancements?

You can join the Machine Learning Engineer newsletter. Join over 70,000 ML professionals and enthusiasts who receive weekly curated articles & tutorials on production Machine Learning.
Also check out the Awesome Production Machine Learning List, where we aim to provide a curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning.

Main Content

Agent Development

• AgentOps - AgentOps helps developers build, evaluate, and monitor AI agents from prototype to production.
• Agents - Agents allows users to build AI-driven server programs that can see, hear, and speak in realtime.
• AgentScope - AgentScope is a multi-agent platform designed to empower developers to build multi-agent applications with large-scale models.
• AgentStack - AgentStack scaffolds your agent stack.
• AIOpsLab - AIOpsLab is a holistic framework to enable the design, development, and evaluation of autonomous AIOps agents..
• AutoGen - AutoGen is an open-source framework for building AI agent systems.
• Chidori - Chidori is a reactive runtime that supports building robust AI agents using languages like Node.js, Python, and Rust, with a focus on reactivity and observability in agent workflows.
• Composio - Composio equip’s your AI agents & LLMs with 100+ high-quality integrations via function calling.
• Concordia - Concordia is a library to facilitate construction and use of generative agent-based models to simulate interactions of agents in grounded physical, social, or digital space.
• CrewAI - CrewAI is a cutting-edge framework for orchestrating role-playing, autonomous AI agents.
• Eko - Eko is a production-ready JavaScript framework that enables developers to create reliable agents, from simple commands to complex workflows.
• IntellAgent - IntellAgent is an advanced multi-agent framework that transforms the evaluation and optimization of conversational agents.
• kagent - kagent is a Kubernetes native framework for building AI agents.
• LangGraph - LangGraph is a library for building stateful, multi-actor applications with LLMs, used to create agent and multi-agent workflows.
• Modelscope-Agent - Modelscope-Agent is a customizable and scalable agent framework.
• OpenAGI - OpenAGI is used as the agent creation package to build agents for AIOS.
• PydanticAI - PydanticAI is a Python agent framework designed to make it less painful to build production grade applications with Generative AI.
• Swarm - Swarm is an educational framework exploring ergonomic, lightweight multi-agent orchestration.
• Swarms - Swarms is an enterprise grade and production ready multi-agent collaboration framework that enables you to orchestrate many agents to work collaboratively at scale to automate real-world activities.
• TensorZero - TensorZero is an open-source framework for building production-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluations, and experimentation.

Context Management

• A2A - Agent2Agent (A2A) protocol addresses a critical challenge in the AI landscape: enabling gen AI agents, built on diverse frameworks by different companies running on separate servers, to communicate and collaborate effectively - as agents, not just as tools.
• agents.json - The agents.json Specification is an open specification that formally describes contracts for API and agent interactions, built on top of the OpenAPI standard.
• ANP - AgentNetworkProtocol (ANP) is an open-source communication standard designed to enable seamless connectivity and collaboration between intelligent agents, positioning itself as the foundational protocol for agent-to-agent interactions in the emerging AI ecosystem.
• MCP Inspector - MCP inspector is a developer tool for testing and debugging MCP servers.

Memory Management

• MemOS - MemOS is an operating system for Large Language Models (LLMs) that enhances them with long-term memory capabilities. It allows LLMs to store, retrieve, and manage information, enabling more context-aware, consistent, and personalized interactions.

Model Deployment

• GenAI Processors - GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.

Prompt Engineering

• ChainForge - ChainForge is an open-source visual programming environment for battle-testing prompts to LLMs. Compare across models, prompts, and prompt parameters using built-in visualizations.
• DSPy - DSPy is the framework for programming—not prompting—language models. It allows you to iterate fast on building modular AI systems and offers algorithms for optimizing prompts and weights.
• ell - ell is a language model programming library that treats prompts as programs. Features automatic versioning, serialization, and rich tooling for prompt engineering with Ell Studio for visualization.
• Latitude - Latitude is the open-source prompt engineering platform to build, evaluate, and refine prompts with AI. Features prompt management, playground testing, AI gateway, and evaluations.
• PromptIDE - PromptIDE by xAI is an integrated development environment for prompt engineering and interpretability research, providing transparent access to Grok-1 with rich analytics and Python SDK support.
• PromptLayer - PromptLayer helps developers maintain a log of their prompts and LLM API requests. Track, debug, and replay completions with a middleware approach.
• PromptSource - PromptSource is a toolkit for creating, sharing and using natural language prompts. Contains a growing collection of prompts (P3: Public Pool of Prompts) written in Jinja templating language.
• Prompty - Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for AI applications. An asset class and format for LLM prompts designed to enhance observability, understandability, and portability.

Visual Engine

• Chat UI - Chat UI is an open-source web application framework that provides the frontend interface and backend infrastructure for building conversational AI chatbots, serving as the codebase behind their HuggingChat platform.
• ComfyUI - ComfyUI is a node-based interface and inference engine for generative AI, specifically designed to work with Stable Diffusion and other AI models. It allows users to create complex workflows for image, video, and other content generation through a visual, graph-like interface.