Synthetic Spanish–English Code-Switching Speech (DS-3)

DS-3 — Synthetic Spanish–English code-switching speech (synthetic voices only)

Created by participant inyongkhafid for the Lost in Transcription — Spanish–English challenge (Mozilla Data Collective / DrivenData, 2026) and published on MDC because the competition requires external training data to be published on MDC. License: GPL-3.0-or-later.

This version contains no voices of real people. All voices were designed from text descriptions with a TTS model. At MDC's request (review of 5 October 2026), the voice-converted copy that was used for training in the competition is not distributed. It is described below in "Voice conversion in the competition training copy (not distributed)".

Summary

DS-3 is a fully synthetic corpus of informal Spanish–English code-switched speech. 86 % of utterances contain both Spanish and English function words. It was built to reduce the mismatch between the Bangor Miami corpus (mostly English, little intra-utterance switching) and conversational voice-note style speech. A voice-converted copy of it (see below) was used, together with the Bangor Miami corpus, to fine-tune Whisper large-v3, Parakeet-TDT-0.6B-v3, Cohere Transcribe 03-2026 and Qwen3-Omni-30B-A3B in the Spanish–English submission that placed 1st on the competition's private leaderboard.

Size

• 2,233 utterances · 5.78 hours · 66,599 words (median 29 words per utterance)
• Duration 1.2–24.9 s (median 9.0 s): < 5 s: 321 · 5–10 s: 993 · 10–20 s: 886 · 20–29.5 s: 33
• 231 synthetic conversations · 2 turn roles (A 1,141 · B 1,092) · 62 synthetic TTS voice personas
• target_voice (used only by the voice-conversion recipe): 37 opaque IDs, one per adult Bangor Miami speaker (sastre 869 · herring 568 · zeledon 422 · maria 374 utterances by corpus family)

Creation pipeline (code to be released with the competition solution under MPL-2.0)

Voice conversion in the competition training copy (not distributed)

For the competition, each clip of this release was processed further before training:

1. Voice conversion with kNN-VC (bshall/knn-vc, MIT, © 2023 MediaLab, Stellenbosch University; WavLM-Large features, MIT; prematched HiFi-GAN vocoder; top-k = 4). Each clip was converted toward one adult speaker of the Bangor Miami corpus (GPL-3.0-or-later; BangorTalk, also on MDC cmmfulo4r018bnz07py4q9t09). The matching set of each speaker came from that speaker's own clean utterances in the corpus (at least 1.5 s long, no overlapping speech, training sessions only), 180–480 s per speaker. Each synthetic persona was mapped to one target speaker by a fixed hash of the persona name, so a persona keeps one voice; the field target_voice gives this assignment as opaque IDs. Words and timing were not changed. The same round-trip QC was applied (2,319 clips kept, QC WER 8.85 %).
2. Acoustic difficulty: speech rate and pause/voiced ratio adjusted toward clip-level timing statistics of conversational voice notes (aggregate statistics of the competition development set, see above), then a synthetic room impulse response (RT60 0.05 + U(0.10–0.35) s), noise/loudness variation and a voice-note codec chain (Opus 24 kbit/s → MP3 64 kbit/s → WAV 16 kHz mono). No external noise corpus. Own code (MPL-2.0) and ffmpeg. This lengthened the training copy to 9.98 hours.
3. Child-voice filter: utterances converted toward child speakers (identified from the CHAT @ID headers) were removed: 86 utterances from 2 speakers. All 37 remaining target voices are adults.

The code that rebuilds the training copy from this dataset and the Bangor Miami corpus will be released with the competition solution. The exact training copy can be provided to the competition organisers for verification.

Structure

ds3_synthetic_es_en_no_vc/
  wav/syn_{conv}_{turn}_{persona}_16k.wav     # 2,233 files, 16 kHz mono
  manifest.jsonl                              # one line per utterance
  LICENSE (GPL-3.0-or-later) · NOTICE · README.md (this datasheet) · SHA256SUMS

Manifest fields: audio (relative path) · transcript (orthographic text) · duration (s) · conversation · turn_role (A/B) · tts_persona · target_voice (opaque ID T01–T37, the same IDs as in the competition training copy).

Orthography

Standard orthography with case, accents and punctuation as generated (e.g. ¿…?, ¡…!). Spanish and English words are written in their own spelling. There are no language tags inside the text. Disfluencies and fillers appear as words when the LLM produced them (≈ 8 per 1,000 words). Numbers are mostly written as words (0.9 % of utterances contain digits).

Samples (synthetic text)

1. syn_0012_01_meksiko0a_16k.wav · 4.2 s — I think the el precio es but it vale la pena maybe you agree
2. syn_0284_06_meksiko2b_16k.wav · 5.2 s — ¿Tú traerás also cheese? ¿me lo dices? anyway now here
3. syn_0001_03_tengah1b_16k.wav · 5.2 s — I think you should but no te olvides to update your résumé porque los recruiters love details.
4. syn_0238_00_meksiko0a_16k.wav · 5.8 s — ¿Sabes que perdí las llaves mami y ahora estoy atrasada porque el metro está lento y no sé si me puedes ayudar para ah tú me...?
5. syn_0092_89_karibia7b_16k.wav · 6.7 s — Sí pienso que el plan es good pero me preocupa la tasa de interés, y el hecho de que no incluyan garantía
6. syn_0331_03_selatan2b_16k.wav · 12.3 s — I can check the circuit board, but primero I need the manual y si no lo tienes we can also call the power company y si no ¿puedes pasarme el número del servicio de energía para llamar?
7. syn_0128_07_tengah7b_16k.wav · 10.4 s — Oye ¿tienes idea si el doctor está en el consultorio ahora but the waiting room es grande y I think we might need to wait y qué te parece si nos encontramos antes de hoy?
8. syn_0334_00_tengah2a_16k.wav · 15.2 s — Hey mijo I saw the shift chart and I think we got schedule today so mi turno es de ocho a diez pero que me preocupa si el día viernes es de doce a diez, I think this might cause a problem for me y si me dan el día libre, podría viajar tarde para la casa de mi abuela de nuevo.

Intended use and limitations

• For ASR training and augmentation. The speech is synthetic: prosody and spontaneity are approximations, and the LLM text can be grammatically odd at switch points.
• Not suitable for linguistic analysis of natural code-switching, for speaker recognition, or for any biometric purpose.
• QC used round-trip ASR (corpus WER 5.35 % on the TTS audio). This number includes the QC model's own recognition errors, so it is an upper bound on label–audio mismatch, not an exact rate.
• Request (not a licence condition): please do not use this dataset, or the recipe above, for speaker identification, voice cloning or impersonation of the Bangor Miami participants, or any other biometric purpose.

Sources and licences

• This release contains no audio from the Bangor Miami corpus. The recipe above uses it: audio produced with the recipe is derived from the corpus and is subject to its terms (GPL-3.0-or-later; BangorTalk does not grant permission to train an AI large language model on the material unless all of that model's training data is made publicly available).
• gpt-oss-20b (Apache-2.0) · Qwen3-TTS-12Hz-1.7B VoiceDesign/Base (Apache-2.0) · llama.cpp (MIT) · recipe only: kNN-VC (MIT), WavLM-Large (MIT).

Citation of the source corpus used by the recipe

Deuchar, M., Davies, P., Herring, J. R., Parafita Couto, M. C., & Carter, D. (2014). Building bilingual corpora. In E. M. Thomas & I. Mennen (Eds.), Advances in the Study of Bilingualism (pp. 93–110). Multilingual Matters.