Datasets | Mozilla Data Collective

A Sindhi–English parallel corpus in JSONL format, designed for machine translation and cross-lingual NLP tasks across diverse real-world domains.

Sindhi to English Parallel Corpus

License: CC-BY-NC-4.0

Locale: snd, eng

Task: MT

Format: JSONL

Size: 2.50 MB

MT benchmark for respecting a provided glossary and translation memory; EN to 9 languages.

Prompsit D5 - Linguistic-resource Adherence

License: CC-BY-4.0

Locale: mul

Task: MT

Format: JSONL

Size: 213.33 KB

MT benchmark for preserving document structure (HTML tree, tables, links) under translation; EN to 9 languages.

Prompsit D4 - Document-structure Integrity

License: CC-BY-4.0

Locale: mul

Task: MT

Format: JSONL

Size: 165.23 KB

MT benchmark for translating only the values of structured resource files (XML/JSON/.properties/ARB); EN to 9 languages.

Prompsit D3 - Structured-resource Integrity

License: CC-BY-4.0

Locale: mul

Task: MT

Format: JSONL

Size: 128.28 KB

MT benchmark for adapting numbers, dates, units and currency to the target locale; en-US to 9 locales.

Prompsit D2 - Locale-data Integrity

License: CC-BY-4.0

Locale: mul

Task: MT

Format: JSONL

Size: 163.87 KB

MT benchmark for preserving inline tags, placeholders and ICU markup under translation; EN to 9 languages.

Prompsit D1 - Inline Asset Integrity

License: CC-BY-4.0

Locale: mul

Task: MT

Format: JSONL

Size: 201.48 KB

Italian, ~25yo male speaking in Neapolitan, a language of sourthern Italy.

Neapolitan TTS - male voice

License: CC-BY-NC-SA-4.0

Locale: nap

Task: TTS

Format: WEBM, TSV, TXT, WAV

Size: 1.53 GB

Italian, ~50yo woman speaking in Ligurian, a language of Liguria, north-east Italy

Ligurian TTS - female voice

License: CC-BY-NC-SA-4.0

Locale: lij

Task: TTS

Format: WAV, WEBM, TSV, TGZ

Size: 328.77 MB

Kaltepetlahtol

Tequila Zongolica Nahuatl ASR Dataset

A 16 hour ASR dataset of Nahuatl from Tequila and Zongolica. Derived from Amith et al (2026)'s field recordings and transcriptions datasets

License: CC-BY-ND-4.0

Locale: nlv

Task: ASR

Format: FLAC, TSV

Size: 875.17 MB

3,272 openly-licensed (Creative Commons) images of Turkish food across 245 dish labels, with documented per-image source, creator, and license.

LezizNet - Turkish food images

License: CC-BY-SA-4.0

Locale: tur

Task: CV

Format: JPG, TSV

Size: 784.29 MB

A single-speaker Ladino (Judeo-Spanish) speech corpus of ~2,000 aligned audio segments for training text-to-speech models.

Ladino TTS Corpus

License: CC-BY-NC-4.0

Locale: lad

Task: TTS

Format: WAV, TSV

Size: 196.00 MB

Field Matters

AIRI AI4TALK Evenki ASR

Evenki ASR data from AIRI AI4TALK with MP3 speech segments and IPA-like transcriptions in CSV.

License: CC-BY-SA-4.0

Locale: evn

Task: ASR

Format: CSV, MP3

Size: 572.87 MB

Habib University

Burushaski-English Speech Translation Corpus

A linguistically informed Burushaski speech - English text corpus containing 14,970 short audio recordings paired with corresponding English text translations.

License: CC-BY-NC-SA-4.0

Locale: bsk

Task: MT

Format: WAV, TXT

Size: 921.47 MB

5 horas de video en Lengua de Señas Mexicana con glosa y traducción al español. Grabado por una persona sorda usuaria de LSM del estado de Querétaro, México.

Corpus paralelo multimodal Lengua de Señas Mexicana – Español MX

License: CC BY-NC-ND 4.0

Locale: mfs, spa

Task: NLP

Format: MOV, TSV

Size: 42.81 GB

iSocial

RAPNIC - Catalan Impaired Speech (Example Set)

An example set of RAPNIC, the first Catalan speech corpus of people with speech disorders such as cerebral palsy and Down syndrome.

License: CC-BY-NC-SA-4.0

Locale: ca

Task: ASR

Format: WAV, TSV

Size: 105.22 MB

Ok nemi totlahtool

Tequila Zongolica Nahuatl Transcriptions

Transcription corpus of Orizaba-Zongolica Nahuatl language (Glottocode:oriz1235), corresponding to approximately 13 hours of audio recordings.

License: CC-BY-ND-4.0

Locale: nlv

Task: ASR

Format: TRS

Size: 342.65 KB

Language-specific StackOverflow vector dataset for Bash.

StackOverflow Vector Dataset - Bash

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 4.40 GB

Language-specific StackOverflow vector dataset for Assembly.

StackOverflow Vector Dataset - Assembly

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 1.35 GB

Language-specific StackOverflow vector dataset for C.

StackOverflow Vector Dataset - C

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 11.07 GB

Language-specific StackOverflow vector dataset for C++.

StackOverflow Vector Dataset - C++

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 20.05 GB

Language-specific StackOverflow vector dataset for Cobol.

StackOverflow Vector Dataset - Cobol

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 192.29 MB

Language-specific StackOverflow vector dataset for C#.

StackOverflow Vector Dataset - C#

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 37.27 GB

Language-specific StackOverflow vector dataset for CSS.

StackOverflow Vector Dataset - CSS

License: CC-BY-SA-4.0

Locale: en-US

Task: RAG

Format: Parquet

Size: 21.96 GB