Center za jezikovne vire in tehnologije Univerze v Ljubljani

Slovenia
www.cjvt.si
Academic

About Center za jezikovne vire in tehnologije Univerze v Ljubljani

The Centre for Language Resources and Technologies at the University of Ljubljana (CJVT UL) is a research unit focusing on scientific research as well as the development and maintenance of key digital language resources and language technology applications for contemporary Slovene. The developed resources and applications have practical value and are accessible to all the Slovene language users around the world.

Datasets

Center za jezikovne vire in tehnologije Univerze v Ljubljani

SI-NLI

SI-NLI (Slovene Natural Language Inference Dataset) contains 5,937 human-created Slovene sentence pairs (premise and hypothesis) that are manually labeled with the labels "entailment", "contradiction", and "neutral". We created the dataset using sentences that appear in the Slovenian reference corpus ccKres (http://hdl.handle.net/11356/1034). Annotators were tasked to modify the hypothesis in a candidate pair in a way that reflects one of the labels. The dataset is balanced since the annotators created three modifications (entailment, contradiction, neutral) for each candidate sentence pair. The dataset is split into train, validation, and test sets, with sizes of 4,392, 547, and 998. We used Slovenian pre-trained language models to create splits, thereby ensuring that difficult and easy instances are evenly distributed in all three subsets. The dataset is released in a tabular TSV format. The README.txt file contains a description of the attributes. Only the hypothesis and premise are given in the test set (i.e. no annotations) since SI-NLI is integrated into the Slovene evaluation framework SloBENCH (https://slobench.cjvt.si/). If you use the dataset to train your models, please consider submitting the test set predictions to SloBENCH to get the evaluation score and see how it compares to others.

License: CC-BY-NC-SA-4.0

Locale: sl

Task: NLU

Format: TSV

Size: 392.44 KB