What changed in ElevenLabs’ voice cloning and multilingual audio AI

ElevenLabs has expanded into conversational automation with ElevenAgents, offering 75ms latency via the Eleven Flash v2.5 model. The platform also features the Eleven v3 model for emotional audio production and faces a federal lawsuit regarding unauthorized voiceprint usage.

What changed in ElevenLabs' voice cloning and multilingual audio AI

Mati Staniszewski and Piotr Dabkowski founded ElevenLabs in 2022 after they felt frustration with poorly dubbed American films in Poland. The company reached an $11 billion valuation in February 2026 after its $500 million Series D funding round led by Sequoia Capital. The company now focuses on ElevenAgents, a platform for deploying conversational bots that handle tasks like booking appointments, lead qualification, or processing payments through tool calls. These agents use proprietary models to handle human-like pauses and hesitations during interruptions. They also secured insurance coverage via AIUC-1 certification, which subjects agents to adversarial tests for security, data privacy, and hallucinations. I find the move toward proactive agents, which can use multimodal reasoning to listen to tone or read uploaded files, more useful than simple chatbots. This expansion into automation led many users to predict the downfall of the BPO industry and call centers in India. The Eleven Flash v2.5 model provides 75ms latency for real-time applications like customer support. The platform uses the Model Context Protocol to allow agents to check a CRM mid-conversation. These agents are the first AI voice agent platform to secure insurance coverage based on AIUC-1 readiness. The Senate Joint Economic Committee recently asked Mati Staniszewski about how the company tracks attempts to clone no-go voices following reports of $893 million in AI voice scam losses. The ElevenAgents 2.0 update introduced a revamped turn-taking model, RAG knowledge retrieval for low-latency database access, and the ability to hand off between multiple specialized agents in a single call. These agents rival competitors like Vapi and Retell AI.

Creative audio production

The Eleven v3 model handles 70+ languages and includes Audio Tags like [whispers] or [shouts] for emotional direction. Users can choose between Instant Voice Cloning from a 60-second sample or Professional Voice Cloning using 20 to 30 minutes of audio. I suggest you test short samples before committing to a subscription because the platform uses a credit-based system where regenerating clips consumes your allowance. The Voice Changer performs speech-to-speech conversion by mapping a recording onto a different voice while preserving the original timing and emotion. The Eleven Music suite allows users to generate full tracks or use stem separation to isolate vocals and drums for professional remixing. The Dubbing Studio translates and re-voices content into 30+ languages while preserving the original speaker’s voice and synchronizing lip movements.

Plan Monthly Price Key Feature
Free $0 10,000 credits
Starter $5 Instant voice cloning
Creator $22 Professional voice cloning
Pro $99 500,000 characters
Scale $499 High volume

The Voice Changer offers two models. The eleven_english_sts_v2 model is for sharper English output, and the eleven_multilingual_sts_v2 model is for cross-language workflows. Using the Segmind endpoint, the cost remains $0.00375 per second of output duration. The library contains over 5,000 voices. Scribe v2 provides real-time speech-to-text transcription across dozens of languages. For creators working in noisy environments, the Voice Isolator strips background noise and ambient sound from recordings to produce clean audio.

Legal and data management

ElevenLabs pays creator rewards exceeding $11 million to those who supply voices for its marketplace. The Iconic Marketplace includes licensed voices from figures like Liza Minnelli and the estates of Maya Angelou and Judy Garland. However, the company faces a federal lawsuit in Chicago filed in May 2026, where seven Illinois residents allege ElevenLabs took voiceprints from third-party platforms without written notice or release to build their commercial models for the entire company.

You should check your data settings if you want to prevent your audio from being used for model improvement. On Free, Creator, Pro, and Scale plans, the company opts users into training by default. Enterprise customers have training turned off by default and can access Zero Retention Mode. The platform operates for 41% of the Fortune 500. The company also manages risks regarding misuse, such as a Russian-linked influence campaign and a robocall impersonating President Biden in New Hampshire. Cloned voices stay in the account and cannot be exported as files. The Eleven Multilingual v2 model supports 29 languages, including Hindi, Arabic, Mandarin, and European Portuguese. The Dubbing Studio supports 30+ languages and processes content from files or YouTube, TikTok, and X URLs. The platform uses usage-based billing for its API on all paid plans. Can the platform eventually solve the legal risks of unauthorized voice imitation?

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories