Datasets | Mozilla Data Collective

A diverse Hindi text corpus spanning literary works and news article blogs, curated for NLP research and language modeling tasks.

Hindi Literature & News Corpus

License: CC-BY-NC-SA-4.0

Locale: hin

Task: NLP

Format: TXT

Size: 19.52 MB

MDC Community Concierge

VoxForge - Persian

16 minutes (157 utterances) of read speech of ‭فارسی‬ (Persian).

License: GPL-3.0

Locale: fa

Task: ASR

Format: WAV, TSV

Size: 58.20 MB

Tamir News Agency

Tamir Sindhi News Corpus

The corpus contains 1.1 million tokens from the Tamir Sindhi Newspaper published from the year 2022-2025. The text consists of the complete newspaper content including headlines, editorials, finance news and advertisements. The newspaper published in Karachi, Pakistan on daily basis.

License: CC-BY-NC-SA-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 2.56 MB

Kaler Kontho

Kaler Kantho Bengali Newspaper Corpus

A 10+ million token Bengali newspaper corpus from Kaler Kantho provided in .docx format for large-scale NLP research and linguistic analysis.

License: CC-BY-NC-4.0

Locale: ben

Task: NLP

Format: DOCX

Size: 33.11 MB

Hindi monolingual corpus of about 10 million tokens, compiled from multiple authors and including literary works and personal articles.

Hindi 10 Million Text Corpus

License: CC-BY-ND-4.0

Locale: hin

Task: NLP

Format: DOCX

Size: 21.91 MB

Text corpus of 176,843 words compiled from 397 Judeo-Espanyol articles from Şalom newspaper.

Şalom Ladino Corpus

License: CC-BY-4.0

Locale: lad

Task: LM

Format: TXT

Size: 403.16 KB

Longitudinal Pakistani Pashto news corpus from Radio Mashaal (2010-2026) with over 52,000 articles and 14.5M tokens.

RFE/RL Pashto (Pakistani) News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: ps

Task: NLP

Format: TXT

Size: 39.26 MB

A 10.4-million-word Saraiki news corpus from Associated Press of Pakistan, covering 2016–2026 for non-commercial Saraiki NLP research.

Associated Press Pakistan 10.4 Million Saraiki Corpus

License: CC-BY-NC-SA-4.0

Locale: skr

Task: NLP

Format: TXT

Size: 19.51 MB

Forum for Language Initiatives

Kohistani Shina Word List

A UTF-8 encoded Kohistani Shina dictionary and word list corpus with lexical entries, meanings, and annotations for linguistic research and NLP use.

License: CC-BY-NC-4.0

Locale: plk

Task: NLP

Format: TXT

Size: 394.05 KB

A compilation of monolingual and parallel Tamazight language datasets created by CIEMEN as part of the Awal community language technology project.

Awal Tamazight Dataset

License: CC-BY-4.0

Locale: zgh

Task: LM

Format: TSV, JSON, TXT

Size: 11.57 MB

Jamuna Printing and Publishing Ltd.

The Daily Jugantor Bengali Language Corpus

Monolingual Bengali corpus of 10.6 million words from Daily Jugantor, a major Bengali news source.

License: CC-BY-NC-SA-4.0

Locale: ben

Task: NLP

Format: DOCX

Size: 40.49 MB

A Balti literature corpus preserving literary, cultural, and linguistic heritage for language documentation, research and low-resource NLP applications.

Balti Literature corpus

License: CC-BY-NC-SA-4.0

Locale: bft

Task: NLP

Format: TXT, DOCX

Size: 292.67 KB

Longitudinal Afghan Dari news corpus from Radio Azadi (2000-2026) featuring over 212,000 articles and 50M tokens.

RFE/RL Afghan Dari News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: prs

Task: NLP

Format: TXT

Size: 115.70 MB

Tamazight NLP

Tamazight Open Speech Dataset

This dataset provides a parsed, formatted, and ready-to-use Amazigh Voice Dataset. It contains voice recordings and corresponding text transcripts in Standard Moroccan Amazigh (ⵜⴰⵎⴰⵣⵉⵖⵜ ⵜⴰⵏⴰⵡⴰⵢⵜ ⵜⴰⵎⵓⵔⴰⴽⵓⵛⵜ) intended for training Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) models.

License: Apache-2.0

Locale: zgh

Task: ASR

Format: WAV, JSONL

Size: 459.09 MB

Longitudinal Turkmen & Russian news corpus from Azatlyk Radiosy (2009-2026) with nearly 65,000 articles and 16.5M tokens.

RFE/RL Turkmen News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: tk,ru

Task: NLP

Format: TXT

Size: 48.28 MB

A Ormuri dictionary by Rozi Khan Burki documenting Ormuri vocabulary and meanings to support language preservation, linguistic research, and NLP development.

Ormuri Bilingual Dictionary by Rozi Khan Burki

License: CC-BY-ND-4.0

Locale: oru

Task: TTS

Format: JPG, PDF, DOCX

Size: 15.68 MB

MDC DatasetsCompensated

🚧 Khowar Corpus for Speech Recognition 🚧

A Khowar ASR dataset with approximately 10 hours of audio with transcriptions.

License: MDC DLA-1.0

Locale: khw

Task: ASR

Format: MP3, TSV

Size: 245.00 MB

A Sindhi–English parallel corpus in JSONL format, designed for machine translation and cross-lingual NLP tasks across diverse real-world domains.

Sindhi to English Parallel Corpus

License: CC-BY-NC-4.0

Locale: snd, eng

Task: MT

Format: JSONL

Size: 2.50 MB

A collection of annotated images capturing public space activities, traditional handicrafts, historical and cultural heritage exhibits in Java.

JAV IKRAR - Craft and Cultural Heritage Multimodal Corpus

License: CC-BY-NC-SA-4.0

Locale: jav

Task: CV

Format: JPEG, WEBM, JSON

Size: 1.13 GB

MDC DatasetsCompensated

🚧 Khowar Speech Translation Corpus 🚧

A Khowar-English speech translation dataset with approximately 9.5 hours of Khowar audio with English text translations.

License: MDC DLA-1.0

Locale: khw, en

Task: MT

Format: MP3, TSV

Size: 244.96 MB