Abstract:
This work addresses the challenge of classifying public sector organizations across multiple European languages using only their official names, a critical step for entity disambiguation in knowledge graph population. We employ ontology-based knowledge extraction to evaluate three natural language processing approaches: rule-based keyword extraction, zero-shot inference, and embedding-based semantic similarity —under low-context, low-resource assumptions. Large language models are integrated across all three techniques. Our methodology systematically evaluates multilingual preprocessing, various state-of-the-art models, different supervision regimes, classification structures, and parameter optimization. We conduct a detailed evaluation across three specific domains (healthcare, administration, education) spanning all European Union countries, analyzing performance in relation to lexical structure and class balance.
Results show embedding-based methods to be the strongest overall: supervised contrastive fine-tuning of the multilingual-e5 model yields the highest mean F1, while cosine similarity to class-label embeddings provides a competitive zero-training baseline. Rule-based methods remain close behind, with TF-IDF keyword selection as the best configuration. Uniquely interpretable, this method provides an analysis of regional naming conventions and lexical features. Zero-shot natural language inference however, does not reach practical performance, unable to discriminate between semantically proximate class labels, and being sensitive to threshold calibration under class imbalance. The consistent gap between lexical and embedding configurations indicates that organization class is largely explained by naming keywords, but with a residual semantic component that lexical matching cannot capture completely.
This work highlights the feasibility of ontology-guided model training from short texts and which can prove valid approaches to entity disambiguation challenges in formal knowledge representation systems, particularly when integrating diverse European organizational entities into structured knowledge bases.
