Urdu-Sindhi Grapheme-to-Phoneme Dataset

Urdu-Sindhi Grapheme-to-Phoneme Dataset

The Urdu-Sindhi Grapheme-to-Phoneme Dataset is a bilingual linguistic resource containing Urdu and Sindhi words paired with their phonetic pronunciations in IPA notation.

The dataset was created by collecting and processing language entries with available IPA pronunciations from Wiktionary, with the goal of providing a structured resource for low-resource Urdu and Sindhi speech and language technologies.

Dataset Composition

The combined dataset contains 130,333 grapheme-to-phoneme records:

• Urdu: 30,657 records
• Sindhi: 99,676 records

The dataset is distributed in Parquet format.

Dataset Structure

urdu_sindhi_g2p/
├── urdu-g2p/
│   ├── train.parquet
│   ├── validation.parquet
│   └── test.parquet
└── SND_G2P/
    └── train.parquet

Urdu

The Urdu portion contains:

• graphemes — Urdu written form
• phonemes — corresponding phonetic representation

Splits:

• Train
• Validation
• Test

Sindhi

The Sindhi portion contains:

• Grapheme — Sindhi written form
• Phoneme — corresponding IPA pronunciation

Split:

• Train

Examples

Urdu Grapheme-to-Phoneme Examples

Sindhi Grapheme-to-Phoneme Examples

Understanding the Data

A grapheme represents the written form of a word, while a phoneme represents its pronunciation.

For example:

Urdu

Grapheme: دعا
Phoneme:  dˈʊaː

and:

Sindhi

Grapheme: آسمان
Phoneme: aː s m aː n ʊ

These mappings can be used as supervised examples for training G2P models that learn to convert written language into phonetic representations.

Data Collection

The dataset was collected from Wiktionary entries containing IPA pronunciation information.

For Urdu, entries were collected from Wiktionary's category for Urdu terms with IPA pronunciation. This category contains Urdu terms for which pronunciation is provided in IPA form.

For Sindhi, entries were collected from Wiktionary's category for Sindhi terms with IPA pronunciation. The collection process identifies Sindhi entries containing pronunciation information and extracts the written term together with its corresponding IPA representation.

The collected entries were then transformed into structured grapheme-to-phoneme pairs.

Data Processing

The processing pipeline consisted of:

1. Identifying Urdu and Sindhi Wiktionary entries containing IPA pronunciation.
2. Extracting the written word or grapheme.
3. Extracting the corresponding IPA pronunciation.
4. Removing unrelated dictionary information.
5. Converting the extracted records into structured tabular data.
6. Separating the data by language.
7. Creating train, validation, and test splits for the Urdu component.
8. Storing the resulting datasets in Parquet format.
9. Packaging the language-specific Parquet files into a single .tar.gz archive.

Only the grapheme-to-phoneme data required for the dataset is included in the released archive.

Phonetic Representation

The pronunciation fields use IPA-style phonetic notation. Depending on the source entry, representations may contain:

• IPA consonants
• IPA vowels
• Long-vowel markers
• Nasalization
• Stress marks
• Affricates
• Diacritics
• Other phonological symbols

The pronunciation representation follows the information available in the corresponding Wiktionary entries.

Dataset Statistics

Intended Applications

The dataset is intended to support non-commercial and academic research in:

• Grapheme-to-Phoneme (G2P) conversion
• Text-to-Speech (TTS)
• Automatic Speech Recognition (ASR)
• Pronunciation modeling
• Speech synthesis
• Multilingual speech technology
• Computational phonology
• Urdu NLP
• Sindhi NLP
• Low-resource language research

Data Quality

The dataset is derived from community-maintained Wiktionary entries. Consequently, pronunciation coverage and accuracy may vary between words and languages.

Users should validate the phonetic representations for their intended downstream application before using the dataset in production systems.

The dataset should be considered a structured research resource rather than a guaranteed authoritative pronunciation dictionary.

Source

The primary source for the linguistic information is Wiktionary.

Urdu

Wiktionary category:

Category:Urdu terms with IPA pronunciation

Sindhi

Wiktionary category:

Category:Sindhi terms with IPA pronunciation

The source entries are community-maintained and may be updated over time.

License

The underlying Wiktionary textual content is available under the Creative Commons Attribution-ShareAlike (CC BY-SA) licensing terms.

Recommended license: CC-BY-SA-4.0

Users should review the applicable Wiktionary licensing terms and attribution requirements before redistributing or publishing derivatives of this dataset.

Attribution

This dataset was constructed from linguistic information available on Wiktionary.

Please attribute the underlying source to:

Wiktionary contributors

Source:

https://en.wiktionary.org/

Urdu source category:

https://en.wiktionary.org/wiki/Category:Urdu_terms_with_IPA_pronunciation

Sindhi source category:

https://en.wiktionary.org/wiki/Category:Sindhi_terms_with_IPA_pronunciation

Release Format

The complete dataset is distributed as:

urdu_sindhi_g2p.tar.gz

The archive contains only the dataset files:

urdu_sindhi_g2p/
├── urdu-g2p/
│   ├── train.parquet
│   ├── validation.parquet
│   └── test.parquet
└── SND_G2P/
    └── train.parquet

No source-control metadata, cache files, hidden files, or unrelated auxiliary files are included in the release archive.

Disclaimer

The dataset is provided for research and educational purposes. Users are responsible for verifying pronunciation data and ensuring that their use, redistribution, and derivative works comply with the applicable source licenses and terms.

The maintainers of this dataset do not claim ownership of the underlying Wiktionary contributions.