efectividad

The World Wide Web is a very large distributed digital information space. From its origins in 1991 as an organization-wide collaborative environment at CERN for sharing research documents in nuclear physics, the Web has grown to encompass diverse information resources: personal home pages; online digital libraries; virtual museums; product and service catalogs; government information for public dissemination; research publications; and Gopher, FTP, Usenet news, and mail servers. Some estimates suggest that the Web currently includes about 150 million pages and that this number doubles every four months. for representing document content.

The goal of this study was to investigate the retrieval effectiveness of three popular German Web search services. For this purpose the engines Altavista.de, Google.de and Lycos.de were compared with each other in terms of the precision of their top twenty results. The test panelists were based on a collection of fifty randomly selected queries, and relevance assessments were made by independent jurors. Relevance assessments were acquired separately a) for the search results themselves and b) for the result descriptions on the search engine results pages. The basic findings were: 1.) Google reached the best result values. Statistical validation showed that Google performed significantly better than Altavista, but there was no significant difference between Google and Lycos. Lycos also attained better values than Altavista, but again the differences reached no significant value. In terms of top twenty precision, the experiment showed similar outcomes to the preceding retrieval test in 2002. Google, followed by Lycos and then Altavista, still performs best, but the gaps between the engines are closer now. 2.) There are big deviations between the relevance assignments based on the judgement of the results themselves and those based on the judgements of the result descriptions on the search engine results pages.

Although relevance has represented a key concept in the field of information science for evaluating information retrieval effectiveness, the broader context established by interdisciplinary frameworks could provide greater depth and breadth to on-going research in the field. This work provides an overview of the nature of relevance in the field of information science with a cursory view of how cross-disciplinary approaches to relevance could represent avenues for further investigation into the evaluative characteristics of relevance as a means for enhanced understanding of human infor-mation behavior.

Es bien sabido que los motores de búsqueda y los índices han surgido como herramienta para ayudar a encontrar información en el enorme y rápidamente creciente volumen de páginas web: su origen se inscribe en el contexto académico, al igual que ocurrió con la propia Red, para pasar posteriormente al escenario comercial.

Se describe y analiza el acceso y la recuperación de información en la World Wide Web. En primer lugar, se estudia el funcionamiento de los motores de búsqueda y los metabuscadores. En segundo lugar, se presenta un estado de situación de la recuperación de información web a través de tríada sistema-documento-usuario. Y por último, se evalúa el desempeño de motores de búsqueda y metabuscadores en dos preexperimentos. El primero, analiza, emulando la conducta del usuario, la ordenación por relevancia de los primeros diez y veinte resultados de cinco motores de búsqueda de mayor cobertura en respuesta a expresiones de búsqueda no estructuras. Y el segundo, calcula las medidas tradicionales de precisión y exhaustividad, determina la proporción de URLs solapados y similitud entre cinco buscadores y tres metabuscadores usando para su interrogación palabras poco frecuentes.

The performance evaluation of an information retrieval system is a decisive aspect for the measure of the improvements in search technology. Our work intends to provide a framework to compare and contrast the performance of search engines in a research environment. In this way, we have designed and developed USim, a tool for the performance evaluation of Web IR systems based on the simulation of users’ behavior. This simulation tool contributes in the performance evaluation process in two ways: estimating the saturation threshold of the system and in the comparison of different search algorithms or engines. The latter point is the most interesting because, as we demonstrated, the comparison using different workload environments will achieve more accurate results (avoiding erroneous conclusions derived from ideal environments). From a general point of view, USim intends to be an approximation to some new performance evaluation techniques specifically developed for the Internet search engines.

The evaluation of traditional IR systems is performed in an ideal situation, without workload in the IR system, however the Web IR systems are under different workload levels through the time. Therefore, the performance evaluation must be done considering several workload environments. For this purpose, we have designed and developed USim, a tool for the performance evaluation of Web IR systems based on the simulation of users’ behaviour. This work is based on several previous theoretical analyses of the users’ behaviour while using Web search systems. This simulation tool helps in the performance evaluation of the Web IR systems in two different ways: comparing the response times for different search algorithms or engines (calculating the response times for each search engine) and estimating the saturation threshold of the system. The latter point is especially important for guaranteeing an appropriate quality of service to all the users, and therefore, its applications to the e-Business of the IR system are quite significant during the whole life of the system.

This paper proposes a set of measures to evaluate search engine functionality over time. When coming to evaluate the performance of Web search engines, the evaluation criteria used in traditional information retrieval systems (precision, recall, etc.) are not sufficient. Web search engines operate in a highly dynamic, distributed environment, therefore it becomes necessary to assess search engine performance not just at a single point in time, but over a whole period.