Skip to content

Comments

Add ElevenLabs Custom Vocab & Orchestration pipeline#93

Merged
dbrkn merged 11 commits intomainfrom
berkin/add-elevenlabs-customvocab
Jan 14, 2026
Merged

Add ElevenLabs Custom Vocab & Orchestration pipeline#93
dbrkn merged 11 commits intomainfrom
berkin/add-elevenlabs-customvocab

Conversation

@dbrkn
Copy link
Contributor

@dbrkn dbrkn commented Jan 13, 2026

This PR adds

  • ElevenLabs Transcription Pipeline with Custom Vocabulary Support and it's benchmarking results
  • ElevenLabs Orchestration Pipeline

Sample comand to test custom vocab:

openbench-cli evaluate \ 
  --pipeline elevenlabs-transcription \  
  --dataset earnings22-kws-filewise \
  -m wer \
  -m keyword_fscore \
  -m keyword_precision \
  -m keyword_recall \
  --use-keywords

Expected Result

Screenshot 2026-01-14 at 8 59 36 PM

Sample comand to test orchestration:

uv run openbench-cli evaluate \
  --pipeline elevenlabs-orchestration \
  --dataset callhome-english \
  -m wer \
  -m wder \
  -m cpwer

Expected Result

Screenshot 2026-01-14 at 8 54 03 PM

@dbrkn dbrkn requested a review from EduardoPach January 13, 2026 19:00
dberkin1 and others added 4 commits January 14, 2026 01:27
* Add elevenlabs orchestration, elevenlabs engine and elevenlabs diarization

* Fix inference dummy Orchestrationsample and filter empty words in orchestration output for Elevenlabs

---------

Co-authored-by: dberkin1 <[email protected]>
@dbrkn dbrkn changed the title Add ElevenLabs custom vocab pipeline Add ElevenLabs Custom Vocab & Orchestration pipeline Jan 14, 2026
@dbrkn dbrkn marked this pull request as ready for review January 14, 2026 16:09
@dbrkn dbrkn requested a review from EduardoPach January 14, 2026 16:10
@dbrkn dbrkn merged commit 955e578 into main Jan 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants