elevenlabs-agents

ElevenLabs Agents for voice AI applications.

Installation

npx claude-plugins install @jezweb/claude-skills/elevenlabs-agents

Contents

Folders: assets, references, scripts

Files: README.md, SKILL.md

Documentation

Comprehensive skill for building production-ready conversational AI voice agents with ElevenLabs.

Auto-Trigger Keywords

This skill should be used when working with:

• ElevenLabs agents, ElevenLabs conversational AI, ElevenLabs platform
• Voice agent, voice chat, conversational AI, voice interface
• ElevenLabs React, ElevenLabs SDK, @elevenlabs/react, @elevenlabs/client
• ElevenLabs CLI, agents as code, elevenlabs agents
• Agent configuration, agent workflow, agent behavior
• System prompt, turn-taking, conversation flow
• Multi-voice, pronunciation dictionary, voice design
• RAG knowledge base, ElevenLabs RAG, knowledge base agent
• ElevenLabs MCP, MCP tools, Model Context Protocol
• Client tools, server tools, webhook tools, system tools, post-call webhook
• Telephony integration, Twilio ElevenLabs, SIP trunk
• Voice testing, agent testing, scenario testing
• HIPAA voice agent, GDPR compliance, voice compliance
• LLM caching, cost optimization, burst pricing
• React Native voice, Swift voice agent, voice widget

What This Skill Covers

Platform Capabilities (29 Features)

Agent Configuration & Management:

• System prompt engineering (6-component framework)
• Turn-taking modes (Eager/Normal/Patient)
• Workflows (visual builder with nodes and edges)
• Dynamic variables and personalization
• Authentication patterns (public/private/signed URLs)

Voice & Language:

• Multi-voice support (5000+ voices, 31 languages)
• Pronunciation dictionaries (IPA/CMU formats)
• Speed control (0.7x-1.2x)
• Voice design and cloning
• Language presets and auto-detection

Knowledge & Tools:

• RAG (Retrieval-Augmented Generation) with knowledge bases
• 4 tool types: Client tools, Server tools (webhooks), MCP tools, System tools
• Tool parameter schemas and validation

SDKs & Integration:

• React SDK (@elevenlabs/react@0.12.3)
• JavaScript SDK (@elevenlabs/client@0.12.2)
• React Native SDK (@elevenlabs/react-native@0.5.7)
• Swift SDK (iOS/macOS)
• Embeddable widget (@elevenlabs/convai-widget-embed@0.5.5)
• Widget core (@elevenlabs/convai-widget-core@0.5.5)
• Scribe (Real-Time Speech-to-Text) - Beta

Testing & Analytics:

• Scenario testing (LLM-based evaluation)
• Tool call testing
• Load testing
• Conversation analysis and data collection
• Analytics dashboard (resolution rates, sentiment, compliance)

Privacy & Compliance:

• Data retention policies (GDPR: 2 years, HIPAA: 6 years)
• Encryption (TLS 1.3, AES-256)
• Regional compliance (US/EU/India)
• SOC 2 compliance

Cost Optimization:

• LLM caching (up to 90% savings on cached inputs)
• Model swapping (GPT/Claude/Gemini)
• Burst pricing (3x concurrency at 2x cost)

DevOps & Advanced:

• CLI (“agents as code”) with multi-environment support
• CI/CD integration (GitHub Actions examples)
• Events (WebSocket/SSE real-time streaming)
• Custom models (bring your own LLM)
• Post-call webhooks
• Chat mode (text-only)
• Telephony integration (Twilio, SIP)

Errors Prevented

This skill prevents 27+ common errors:

1. Package deprecation (@11labs/* → @elevenlabs/*)
2. Android audio cutoff (connectionDelay configuration)
3. CSP violations (workletPaths self-hosting)
4. WebRTC vs WebSocket confusion (different auth flows)
5. Missing required dynamic variables
6. Case-sensitive tool names mismatch
7. Webhook authentication failures (HMAC verification, header name)
8. Voice consistency issues (training data quality)
9. Wrong language voice (English voice for Spanish, etc.)
10. Restricted API keys in CLI
11. Agent configuration push conflicts
12. Tool parameter schema mismatches
13. RAG index not ready before use
14. WebSocket protocol errors (1002)
15. 401 Unauthorized in production (visibility settings)
16. Allowlist connection errors
17. Workflow infinite loops
18. Webhook null message on tool calls (message: z.string().nullable())
19. Webhook call_successful is string (“success” not true)
20. Webhook undocumented fields (agent_metadata, llm_usage, etc.)
21. MCP server SSE responses don’t work - ElevenLabs says “Streamable HTTP” but expects plain JSON (protocol 2024-11-05), NOT SSE (protocol 2025-03-26). Use raw Hono+JSON-RPC, NOT official MCP SDK.

Quick Start Examples

React SDK (Voice Chat UI)

import { useConversation } from '@elevenlabs/react';

const { startConversation, stopConversation, status } = useConversation({
  agentId: 'your-agent-id',
  apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY,
  onConnect: () => console.log('Connected'),
  onEvent: (event) => console.log('Event:', event)
});

CLI (“Agents as Code”)

elevenlabs agents init
elevenlabs agents add "Support Agent" --template customer-service
elevenlabs agents push --env prod

API (Programmatic Ag

…(truncated)

Source

View on GitHub