Datasets | Mozilla Data Collective

Longitudinal Kazakh news corpus from Radio Azattyq (2003-2026) with over 139,000 articles and 35M tokens.

RFE/RL Kazakh News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: kk

Task: NLP

Format: TXT

Size: 126.81 MB

A literary corpus of Dameli texts preserving the language’s literary traditions, cultural heritage, narratives, and diverse forms of written expression.

Dameli Literature Corpus

License: CC-BY-NC-4.0

Locale: dml

Task: NLP

Format: TXT, DOCX

Size: 9.83 MB

A large-scale Sindhi Part-of-Speech (SiPOS) tagging dataset for developing and evaluating NLP models in a low-resource language.

Sindhi Parts Of Speech (SiPOS)

License: CC-BY-NC-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 874.04 KB

CLEAR Global

Kanuri Books Corpus

Randomized sentences from Kanuri-language books by four authors, containing 10,281 sentences and 90,706 words.

License: CC-BY-4.0

Locale: kr

Task: LM

Format: TXT

Size: 545.68 KB

Forum for Language Initiatives

Kohistani Shina Word List

A UTF-8 encoded Kohistani Shina dictionary and word list corpus with lexical entries, meanings, and annotations for linguistic research and NLP use.

License: CC-BY-NC-4.0

Locale: plk

Task: NLP

Format: TXT

Size: 394.05 KB

A curated Kannada text corpus containing diverse written content for natural language processing, linguistic research, and language technology development.

Kannada Text Corpus

License: CC-BY-NC-SA-4.0

Locale: kan

Task: NLP

Format: TXT

Size: 2.61 MB

Longitudinal Macedonian news corpus from Radio Slobodna Evropa (2002-2026) with over 204,000 articles and 46M tokens.

RFE/RL Macedonian News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: mk

Task: NLP

Format: TXT

Size: 133.95 MB

A Ormuri dictionary by Rozi Khan Burki documenting Ormuri vocabulary and meanings to support language preservation, linguistic research, and NLP development.

Ormuri Bilingual Dictionary by Rozi Khan Burki

License: CC-BY-ND-4.0

Locale: oru

Task: TTS

Format: JPG, PDF, DOCX

Size: 15.68 MB

Pro Svizra Rumantscha

Vallader Newspaper Corpus

6.2 million tokens in the Vallader variety of Romansh from the daily newspaper ”La Quotidiana”.

License: CC0-1.0

Locale: rm-vallader

Task: OTH

Format: TSV

Size: 18.71 MB

A collection of annotated images capturing public space activities, traditional handicrafts, historical and cultural heritage exhibits in Java.

JAV IKRAR - Craft and Cultural Heritage Multimodal Corpus

License: CC-BY-NC-SA-4.0

Locale: jav

Task: CV

Format: JPEG, WEBM, JSON

Size: 1.13 GB

A sentence-level annotated dataset for training and evaluating sequence labeling models such as CRF, BiLSTM, and transformer-based architectures.

Sindhi Word Segmentation (SdSEG)

License: CC-BY-NC-4.0

Locale: snd

Task: NLP

Format: JSON

Size: 1.13 MB

MDC DatasetsCompensated

🚧 Hazargi-English Parallel Text Corpus 🚧

A Hazargi-English parallel corpus derived from transcribed and translated video recordings

License: MDC DLA-1.0

Locale: haz, en

Task: MT

Format: TSV

Size: 434.04 KB

Longitudinal Uzbek news corpus from Radio Ozodlik (2002-2026) with over 166,000 articles and 31M tokens.

RFE/RL Uzbek News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: uz

Task: NLP

Format: TXT

Size: 154.21 MB

Habib University

Burushaski-English Speech Translation Corpus

A linguistically informed Burushaski speech - English text corpus containing 14,970 short audio recordings paired with corresponding English text translations.

License: CC-BY-NC-SA-4.0

Locale: bsk

Task: MT

Format: WAV, TXT

Size: 921.47 MB

MirasAICompensated

🚧 Noakhali 10 Hours TTS (Male Speaker) 🚧

A high-quality Noakhali TTS dataset with paired speech recordings and text transcriptions for speech synthesis, linguistic research, and language preservation.

License: MirasAI Data License

Locale: oak

Task: TTS

Format: WAV, TSV

Size: 2.87 GB

Habib University

Burushaski-English Speech Translation Corpus

A linguistically informed Burushaski speech - English text corpus containing 14,970 short audio recordings paired with corresponding English text translations.

License: CC-BY-NC-SA-4.0

Locale: bsk

Task: MT

Format: WAV, TXT

Size: 921.47 MB

Anjuman e Katib

Anjuman-e-Katib Farsi/Persian Literature Corpus

This corpus is a collection of more than one million tokens of Farsi/Persian language. The corpus contains work of literature including novels, fictional and non-fictional, poems and many more. the data is shared with the approval of the authors and aims to support linguistic research, language technology development and its preservation.

License: CC-BY-NC-4.0

Locale: fas

Task: NLP

Format: TXT

Size: 2.82 MB

TAUSCompensated

🚧 Colloquial Parallel Text (English - Hausa) 🚧

49k native-reviewed colloquial sentences translated from US English into Hausa for fine-tuning conversational MT and AI models.

License: TAUS DUA

Locale: en-US, ha-NG

Task: MT

Format: TSV

Size: 2.93 MB

TAUSCompensated

🚧 Colloquial Parallel Text (English - Hausa) 🚧

100k native-reviewed colloquial sentences translated from US English into Hausa for fine-tuning conversational MT and AI models.

License: TAUS DUA

Locale: en-US, ha-NG

Task: MT

Format: TSV

Size: 4.49 MB

A 5.1-million-word Arabic-language corpus of 41,963 Associated Press Pakistan (APP) news articles (2016–2026).

Associated Press Pakistan 5.1 Million Arabic Corpus

License: CC-BY-NC-SA-4.0

Locale: arb

Task: NLP

Format: TXT

Size: 9.60 MB

A corpus of annotated photographs of cultural objects described in the Gresik dialect of Javanese language.

JAV-Culture: Gresik–Surabaya Multimodal Corpus

License: CC-BY-NC-SA-4.0

Locale: jav

Task: CV

Format: JPEG, WEBM, JSON

Size: 713.73 MB

LocaleNLP

English Hausa Parallel Corpus

An English–Hausa dataset with 5,000 sentence pairs useful for machine translation and basic language processing tasks.

License: CC-BY-NC-4.0

Locale: eng, hau

Task: MT

Format: csv

Size: 164.32 KB

Longitudinal Azerbaijani & Russian news corpus from Radio Azadlıq (2005-2026) with over 239,000 articles and 37M tokens.

RFE/RL Azerbaijani News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: az,ru

Task: NLP

Format: TXT

Size: 211.65 MB