Datasets | Mozilla Data Collective

Kaleem Art Press

Saraiki-English Parallel Corpus

English–Saraiki parallel corpus: 51,447 aligned sentence pairs (~0.89M words), translated by Kaleem Art Press for MT and Saraiki NLP research.

License: CC-BY-NC-4.0

Locale: mul

Task: MT

Format: CSV

Size: 1.92 MB

Baloch Publishers Multan

Baloch Publishers Saraiki Literature Corpus

This corpus is a collection of one million tokens of Saraiki language. The data was produced under the Baloch Publishers over the last ten years. The corpus contains work of literature including dictionaries, short stories, novels, fiction, non-fiction, and travelogue. The data is being shared with the approval of the authors. It aims to support linguistic research, language technology development, and cultural preservation.

License: CC-BY-NC-4.0

Locale: skr

Task: NLP

Format: TXT

Size: 2.04 MB

LocaleNLP

English Hausa Parallel Corpus

An English–Hausa dataset with 5,000 sentence pairs useful for machine translation and basic language processing tasks.

License: CC-BY-NC-4.0

Locale: eng, hau

Task: MT

Format: csv

Size: 164.32 KB

MDC DatasetsCompensated

🚧 Hazargi-English Parallel Text Corpus 🚧

A Hazargi-English parallel corpus derived from transcribed and translated video recordings

License: MDC-DLA-1.0

Locale: haz, en

Task: MT

Format: TSV

Size: 434.04 KB

Kaleem Art Press

Kaleem Art Press Saraiki Literature Corpus

This corpus contains approximately one million tokens of Saraiki text curated over the past ten years by Kaleem Art Press. It features a wide range of literary genres, including stories, short stories, novels, fiction, non-fiction, travelogues, poetry, biographies, and historical writings. All content is shared with full author approval. The dataset is intended to support linguistic research, Saraiki language technology development, and the preservation of cultural and literary heritage.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

MEDIAMEN

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

This parallel sentences corpus containing 30,405 aligned sentence pairs with a total of approximately 0.62 million tokens, curated from the archival materials of Mediamen (Advertising Agency). The sentences were professionally translated from English into Punjabi (Shahmukhi) and are intended to support machine translation, linguistic research, and Punjabi language technology development, particularly for real-world and contemporary language use.

License: CC-BY-NC-4.0

Locale: en-PK, pnb

Task: MT

Format: CSV

Size: 1.08 MB

Kaleem Art Press

Saraiki Literature Corpus

This contains multiple Saraiki Language books of Stories, Short Stories, Novel, Travelogue, Sentences and collection of articles.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

Community

Sindhi to English Parallel Corpus

A Sindhi–English parallel corpus in JSONL format, designed for machine translation and cross-lingual NLP tasks across diverse real-world domains.

License: CC-BY-NC-4.0

Locale: snd, eng

Task: MT

Format: JSONL

Size: 2.50 MB

Kaleem Art Press

Multilingual Religious Parallel Corpus (Kaleem Art Press)

This dataset is a multilingual parallel sentences corpus containing 6,465 aligned sentence units with approximately 0.98 million words, curated from Kaleem Art Press archives. It includes parallel religious text data in Arabic, Urdu, Saraiki (standard and dialectal), Punjabi (Shahmukhi), and English, supporting research in machine translation, comparative linguistics, digital humanities, and low-resource language studies.

License: CC-BY-SA-4.0

Locale: mul

Task: MT

Format: CSV

Size: 2.27 MB

MDC DatasetsCompensated

🚧 Khowar-English Parallel Corpus 🚧

A parallel corpus of over 10,000 sentences of Khowar and English. The sentences are derived from a multimodal corpus consisting of 152 transcribed and translated Khowar language videos.

License: MDC-DLA-1.0

Locale: khw, en

Task: MT

Format: TSV

Size: 407.57 KB

MirasAI

Associated Press Pakistan 10.4 Million Saraiki Corpus

A 10.4-million-word Saraiki news corpus from Associated Press of Pakistan, covering 2016–2026 for non-commercial Saraiki NLP research.

License: CC-BY-NC-SA-4.0

Locale: skr

Task: NLP

Format: TXT

Size: 19.51 MB

Habib University

Burushaski-English Speech Translation Corpus

A linguistically informed Burushaski speech - English text corpus containing 14,970 short audio recordings paired with corresponding English text translations.

License: CC-BY-NC-SA-4.0

Locale: bsk

Task: MT

Format: WAV, TXT

Size: 921.47 MB

Habib University

Burushaski-English Speech Translation Corpus

A linguistically informed Burushaski speech - English text corpus containing 14,970 short audio recordings paired with corresponding English text translations.

License: CC-BY-NC-SA-4.0

Locale: bsk

Task: MT

Format: WAV, TXT

Size: 921.47 MB

CLEAR Global

TWB Parallel Sentence kits - Hausa (30k)

30,000 English–Hausa parallel sentences from CLEAR Global's Gamayun kits, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: hau

Task: MT

Format: TSV

Size: 1.68 MB

CLEAR Global

TWB Parallel Sentence kits - Swahili (5k)

5,000 English–Swahili parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: swa

Task: MT

Format: TSV

Size: 347.61 KB

Mozilla Common Voice

Common Voice Scripted Speech 27.0 - Saraiki

A collection of read speech recordings in Saraiki.

License: CC0-1.0

Locale: skr

Task: ASR

Format: MP3

Size: 133.50 MB

Kaleem Art Press

Jhoke Publisher Multan’s Saraiki Newspaper Corpus

Jhoke Publisher Multan’s Saraiki Newspaper Corpus (~1.25M tokens) is a normalized UTF-8 collection of Saraiki newspaper.

License: CC-BY-NC-4.0

Locale: skr

Task: NLP

Format: TXT

Size: 2.30 MB

CLEAR Global

TWB Parallel Sentence kits - Kanuri (5k)

5,000 English–Kanuri parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: kau

Task: MT

Format: TSV

Size: 358.46 KB

Bahasa Bahana

BAHANA: Parallel Corpus IND-ENG-ARB

The dataset covers a parallel corpus of 5,174 lines of literal translation text in Indonesian, English and Arabic.

License: CC-BY-SA-4.0

Locale: ind,eng,arb

Task: MT

Format: TSV

Size: 437.30 KB

Community

Ngapak Dialect of JAV-IND-ENG-ARB Parallel Corpus

This dataset consists of a parallel corpus of Javanese translations of the Ngapak dialect from the city of Tegal in Central Java, Indonesia.

License: CC-BY-SA-4.0

Locale: ind,jav,eng,arb

Task: MT

Format: TSV

Size: 544.29 KB

MirasAI

Saraiki 10 Hours TTS Dataset

A 10-hour Saraiki text-to-speech dataset consisting of recorded speech and aligned transcripts, designed for speech synthesis research and development.

License: CC-BY-NC-SA-4.0

Locale: srk

Task: TTS

Format: WEBM, TSV

Size: 584.44 MB

MDC DatasetsCompensated

🚧 Hazargi-English Speech Translation Corpus 🚧

A Hazargi speech translation dataset with approximately 10 hours of Hazargi audio with English text translations.

License: MDC-DLA-1.0

Locale: haz, en

Task: MT

Format: MP3, TSV

Size: 511.58 MB

Sujaak Adbi Sangat

Saraiki Quarterly Magazine Wasson Wehray Corpus

This corpus contains 11,79,200 tokens, from a Saraiki Quarterly Magazine "Wasson Wehray"

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 2.09 MB

CLEAR Global

TWB Parallel Sentence kits - Rohingya (5k)

5,000 English–Rohingya parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: rhg

Task: MT

Format: TSV

Size: 358.88 KB

All Datasets

Saraiki-English Parallel Corpus

Baloch Publishers Saraiki Literature Corpus

English Hausa Parallel Corpus

🚧 Hazargi-English Parallel Text Corpus 🚧

Kaleem Art Press Saraiki Literature Corpus

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

Weekly dataset digest

Saraiki Literature Corpus

Sindhi to English Parallel Corpus

Multilingual Religious Parallel Corpus (Kaleem Art Press)

🚧 Khowar-English Parallel Corpus 🚧

Associated Press Pakistan 10.4 Million Saraiki Corpus

Burushaski-English Speech Translation Corpus

Burushaski-English Speech Translation Corpus

TWB Parallel Sentence kits - Hausa (30k)

TWB Parallel Sentence kits - Swahili (5k)

Common Voice Scripted Speech 27.0 - Saraiki

Jhoke Publisher Multan’s Saraiki Newspaper Corpus

TWB Parallel Sentence kits - Kanuri (5k)

BAHANA: Parallel Corpus IND-ENG-ARB

Ngapak Dialect of JAV-IND-ENG-ARB Parallel Corpus

Saraiki 10 Hours TTS Dataset

🚧 Hazargi-English Speech Translation Corpus 🚧

Saraiki Quarterly Magazine Wasson Wehray Corpus

TWB Parallel Sentence kits - Rohingya (5k)