Datasets | Mozilla Data Collective

Longitudinal Kazakh news corpus from Radio Azattyq (2003-2026) with over 139,000 articles and 35M tokens.

RFE/RL Kazakh News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: kk

Task: NLP

Format: TXT

Size: 126.81 MB

A 500K token corpus of Rangpuri literature and drama provided in UTF-8 format for linguistic research and NLP development.

Rangpuri (অংপুরি Ôṅgpuri) Text Corpus

License: CC-BY-NC-4.0

Locale: rkt

Task: NLP

Format: TXT

Size: 3.68 MB

Ormuri Literature Corpus is a collection of Ormuri literary texts by Rozi Burki, supporting research and preservation of this under-resourced language.

Ormuri Literature corpus by Rozi Burki

License: CC-BY-NC-SA-4.0

Locale: oru

Task: NLP

Format: PDF, DOCX

Size: 18.92 MB

2.9 million tokens in the Surmiran variety of Romansh from the daily newspaper “La Quotidiana”.

Surmiran Newspaper Corpus

License: CC0-1.0

Locale: rm-surmiran

Task: OTH

Format: TSV

Size: 11.89 MB

PT Pancaran Semangat Jaya

Corpus of Panjebar Semangat Javanese-Language Magazine

This dataset is a TXT-format collection compiled from three years of popular articles published in the Javanese-language weekly magazine Panjebar Semangat. It compiles widely read, non-academic Javanese texts reflecting contemporary themes and language use.

License: CC-BY-SA-4.0

Locale: Jav

Task: OTH

Format: TXT

Size: 4.31 MB

A 690K token corpus of raw Chittagonian text covering drama, poetry, and folklore for linguistic research and NLP development.

Chittagonian (চাটগাঁইয়া, saṭgãia) Text Corpus

License: CC-BY-NC-SA-4.0

Locale: ctg

Task: NLP

Format: TXT

Size: 6.59 MB

14.6 million tokens in the Sursilvan variety of Romansh from the daily newspaper “La Quotidiana”.

Sursilvan Newspaper Corpus

License: CC0-1.0

Locale: rm-sursilv

Task: OTH

Format: TSV

Size: 37.80 MB

A 524K token Sylheti corpus featuring drama scripts and cultural texts in UTF-8 format for linguistic research and NLP development.

Sylheti Text Corpus by Haque Publishers

License: CC-BY-NC-4.0

Locale: syl

Task: NLP

Format: TXT

Size: 3.53 MB

Longitudinal Uzbek news corpus from Radio Ozodlik (2002-2026) with over 166,000 articles and 31M tokens.

RFE/RL Uzbek News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: uz

Task: NLP

Format: TXT

Size: 154.21 MB

A Northern Pashto multimodal dataset with 10 hours of synchronized video, audio, transcriptions, speech segmentation, and ELAN annotations.

🚧 Northern Pashto Multimodal Dataset 🚧

License: MDC DLA-1.0

Locale: pbu

Task: NLP

Format: MP4, WAV, EAF

Size: 251.41 GB

1.3 million tokens in the Putèr variety of Romansh from the daily newspaper “La Quotidiana”.

Putèr Newspaper Corpus

License: CC0-1.0

Locale: rm-puter

Task: OTH

Format: TSV

Size: 8.94 MB

Longitudinal Azerbaijani & Russian news corpus from Radio Azadlıq (2005-2026) with over 239,000 articles and 37M tokens.

RFE/RL Azerbaijani News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: az,ru

Task: NLP

Format: TXT

Size: 211.65 MB

Community

Şalom Ladino Corpus

Text corpus of 176,843 words compiled from 397 Judeo-Espanyol articles from Şalom newspaper.

License: CC-BY-4.0

Locale: lad

Task: LM

Format: TXT

Size: 403.16 KB

Forum for Language Initiatives

Khowar Word List

A UTF-8 encoded Khowar lexical corpus containing alphabet definitions and a structured word list for linguistic research and language technology development.

License: CC-BY-NC-4.0

Locale: khw

Task: NLP

Format: TXT

Size: 64.22 KB

Kaleem Art Press

Saraiki Literature Corpus

This contains multiple Saraiki Language books of Stories, Short Stories, Novel, Travelogue, Sentences and collection of articles.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

LocaleNLP

English Hausa Parallel Corpus

An English–Hausa dataset with 5,000 sentence pairs useful for machine translation and basic language processing tasks.

License: CC-BY-NC-4.0

Locale: eng, hau

Task: MT

Format: csv

Size: 164.32 KB

6.1 million tokens in the Rumantsch Grischun variety of Romansh from the daily newspaper “La Quotidiana”.

Rumantsch Grischun Newspaper Corpus

License: CC0-1.0

Locale: rm-rumgr

Task: OTH

Format: TSV

Size: 19.03 MB

A high-quality single-speaker Urdu text-to-speech (TTS) dataset with 25 hours of paired speech recordings and transcriptions for speech synthesis.

Single Speaker Urdu TTS dataset (Female Speaker, South Punjab)

License: MDC DLA-1.0

Locale: ur

Task: TTS

Format: WAV, TSV

Size: 6.02 GB

A Hazargi ASR dataset with approximately 10 hours of audio with transcriptions.

🚧 Hazargi Corpus for Speech Recognition 🚧

License: MDC DLA-1.0

Locale: haz

Task: ASR

Format: MP3, TSV

Size: 511.56 MB

10-hour Punjabi TTS dataset in Shahmukhi script with paired audio and transcripts, designed for speech synthesis and Punjabi language technology.

Punjabi 10 Hours TTS

License: CC-BY-NC-SA-4.0

Locale: pnb

Task: TTS

Format: WEBM, TSV

Size: 481.96 MB

Gawarbati-language literature corpus featuring Diwan Abu Fazl, a poetry collection for documentation and NLP research.

Gawar-Bati Literature Corpus

License: CC-BY-NC-SA-4.0

Locale: gwt

Task: NLP

Format: TXT, DOCX

Size: 876.83 KB

Tamazight NLP

AmaWar

Amawal Warayni - ⴰⵎⴰⵡⴰⵍ ⴰⵙⵏⵎⴰⵍⴰⵢ ⵏ ⵉⵏⵓⵎⴰⴽ ⵏ ⵡⴰⵙⵙⴰⵖⵏ ⴷ ⵉⵎⵢⴰⴳⵏ ⵏ ⵜⵎⴰⵣⵉⵖⵜ ⵏ ⴰⵢⵜ ⵡⴰⵔⴰⵢⵏ

License: Apache-2.0

Locale: rif

Task: MT

Format: TSV

Size: 100.28 KB

A Hazargi speech translation dataset with approximately 10 hours of Hazargi audio with English text translations.

🚧 Hazargi-English Speech Translation Corpus 🚧

License: MDC DLA-1.0

Locale: haz, en

Task: MT

Format: MP3, TSV

Size: 511.58 MB