Skip to main content

Research Repository

Advanced Search

A new weighting scheme and discriminative approach for information retrieval in static and dynamic document collections

Ibrahim, Osman A. S.; Landa-Silva, Dario

A new weighting scheme and discriminative approach for information retrieval in static and dynamic document collections Thumbnail


Authors

Osman A. S. Ibrahim

Profile Image

DARIO LANDA SILVA DARIO.LANDASILVA@NOTTINGHAM.AC.UK
Professor of Computational Optimisation



Abstract

This paper introduces a new weighting scheme in information retrieval. It also proposes using the document centroid as a threshold for normalizing documents in a document collection. Document centroid normalization helps to achieve more effective information retrieval as it enables good discrimination between documents. In the context of a machine learning application, namely unsupervised document indexing and retrieval, we compared the effectiveness of the proposed weighting scheme to the 'Term Frequency - Inverse Document Frequency' or TF-IDF, which is commonly used and considered as one of the best existing weighting schemes. The paper shows how the document centroid is used to remove less significant weights from documents and how this helps to achieve better retrieval effectiveness. Most of the existing weighting schemes in information retrieval research assume that the whole document collection is static. The results presented in this paper show that the proposed weighting scheme can produce higher retrieval effectiveness compared with the TF-IDF weighting scheme, in both static and dynamic document collections. The results also show the variation in information retrieval effectiveness that is achieved for static and dynamic document collections by using a specific weighting scheme. This type of comparison has not been presented in the literature before.

Citation

Ibrahim, O. A. S., & Landa-Silva, D. (2014). A new weighting scheme and discriminative approach for information retrieval in static and dynamic document collections.

Conference Name 14th UK Workshop on Computational Intelligence (UKCI2014)
End Date Sep 10, 2014
Publication Date Sep 1, 2014
Deposit Date Jan 22, 2016
Publicly Available Date Mar 29, 2024
Peer Reviewed Peer Reviewed
Keywords information retrieval
Public URL https://nottingham-repository.worktribe.com/output/994522
Publisher URL http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6930160&filter%3DAND%28p_IS_Number%3A6930143%29
Additional Information © 2014 IEEE. Published in: 2014 14th UK Workshop on Computational Intelligence (UKCI): Student Central Lecture Theatre SC0.51, University of Bradford, Bradford, West Yorkshire, UK, 8-10 September 2014 / editors Daniel Neagu, Mariam Kiran, Paul Trundle. [Piscataway, N.J.] :IEEE,c2014.
ISBN 9781479955381.

Files





You might also like



Downloadable Citations