Three discovery tools: A comparative analysis of retrieval scope, ranking effectiveness, and topic diversity

Abstract

Discovery tools facilitate access to large-scale academic collections, yet their retrieval performance varies. This study presents a comparative analysis of three discovery tools—EBSCO Discovery Services (EDS), EKUAL Discovery Services (EKUAL DS), and Piri Discovery Services (Piri DS)—evaluating retrieval scope, ranking quality, and topical diversity index. The iSearch test collection, derived from arXiv articles, was used with predefined search queries. To assess coverage, the full arXiv corpus was queried to identify indexing differences. A total of 63 queries were executed, and retrieved lists were analyzed for relevance and ranking distribution. Expert-evaluated relevant articles were used to assess retrieval accuracy. Ranking was measured using Discounted Cumulative Gain (DCG) and Normalized DCG (NDCG), and topical diversity was evaluated using the Shannon Diversity Index. EDS and EKUAL DS retrieved identical results, while Piri DS retrieved fewer records, affecting its retrieval completeness. Piri DS ranked relevant articles higher, but with broader distribution. While all tools exhibited comparable ranking performance, EDS and EKUAL DS demonstrated greater topical diversity. These findings offer empirical insight into the strengths and limitations of discovery tools and support libraries in improving search efficiency and retrieval strategies.