READMEmain

Luna: Voice-Enabled Expense Management Agent

Luna is a voice-enabled AI assistant built with Langgraph that helps users manage their expenses through natural conversation. This project demonstrates how to create a voice interface for any Langgraph agent, combining speech-to-text and text-to-speech capabilities with a powerful agent framework.

🌟 Features

Voice Interaction: Speak to Luna and hear responses through high-quality text-to-speech
Expense Management: Create, query, update, and delete expenses through natural conversation
Category Classification: Automatically categorizes expenses based on descriptions
Database Integration: Stores expense data in a PostgreSQL database (via Supabase)
Tool-using Agent: Built with Langgraph's agent framework for complex reasoning

🛠️ Technology Stack

Backend

Python 3.13: Core language for the backend
Langgraph: Agent framework for building the conversational AI
OpenAI:
- Whisper API for speech-to-text
- GPT-4 Mini for the agent's reasoning
- TTS API for text-to-speech responses
MCP (Model Calling Protocol): For defining and using tools
SQLAlchemy: ORM for database interactions
Supabase: PostgreSQL database provider

Audio Processing

sounddevice: For capturing audio from microphone
scipy: For audio file processing

📋 Prerequisites

Python 3.13
OpenAI API key
Supabase account and database
Microphone and speakers

🚀 Getting Started

1. Clone the repository

git clone https://github.com/rosiefaulkner/langgraph-voice-agent.git
cd langgraph-voice-agent

2. Set up a virtual environment and install dependencies

(Recommended) use uv for dependency management

Setup the venv in your project directory and install all dependencies with one command.

uv sync

3. Set up environment variables

Create a .env file in the root directory with the following variables:

OPENAI_API_KEY=your_openai_api_key
SUPABASE_URI=postgresql://postgres:password@db.example.supabase.co:5432/postgres

4. Run the application

python main.py

🎤 Using Luna

Run the application
When prompted, speak your request (e.g., "Create a new expense for lunch today that cost $15")
Press Enter to stop recording
Luna will process your request, interact with the database if needed, and respond verbally
Continue the conversation or say "exit" or "quit" to end the session

🧩 Project Structure

langgraph-voice-agent/
├── main.py                  # Main application entry point
├── assistant_graph.py       # Langgraph agent definition
├── state.py                 # State management for the agent
├── voice_utils.py           # Audio recording and playback utilities
├── mcps/                    # Model Calling Protocol servers
│   ├── mcp_config.json      # MCP server configuration
│   └── local_servers/
│       └── db.py            # Database tools implementation
├── .env                     # Environment variables (not in repo)
├── .env.example             # Example environment variables
└── pyproject.toml           # Project dependencies

🔧 Customizing the Agent

Modifying the System Prompt

To change Luna's personality or capabilities, edit the system_prompt in assistant_graph.py:

system_prompt = """You are Luna, the company's expense manager...

Adding New Tools

Create a new MCP server or add tools to the existing one in mcps/local_servers/
Register the server in mcps/mcp_config.json
The tools will be automatically available to the agent

Changing Voice Settings

Modify the TTS settings in voice_utils.py:

async def play_audio(message: str):
    # ...
    async with openai_async.audio.speech.with_streaming_response.create(
        model="gpt-4o-mini-tts",
        voice="fable",  # Change the voice here
        input=cleaned_message,
        instructions="Speak in a cheerful, helpful tone with a brisk pace.",  # Modify instructions
        response_format="pcm",
        speed=1.2,  # Adjust speed
    ) as response:
        # ...

📚 Resources

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

langgraph-voice-agent