Browsing by Author "Torresani, Lorenzo"
Now showing 1 - 3 of 3
Results Per Page
Sort Options
Item Improving tag transfer for image annotation using visual and semantic information(IEEE Computer Society, 2014) Rodriguez-Vaamonde, Sergio; Torresani, Lorenzo; Espinosa, Koldo; Garrote, Estibaliz; Tecnalia Research & InnovationThis paper addresses the problem of image annotation using a combination of visual and semantic information. Our model involves two stages: a Nearest Neighbor computation and a tag transfer stage that collects the final annotations. For the latter stage, several algorithms have been implemented in the past using labels' information or including implicitly some visual features. In this paper we propose a novel algorithm for tag transfer that takes advantage explicitly of semantic and visual information. We also present a structured training procedure based on a concept we have called Image Networking: all the images in a training database are 'connected' visually and semantically, so it is possible to exploit these connections to learn the tag transfer parameters at annotation time. This learning is local for the test image and it exploits the information obtained in the Nearest Neighbor computation stage. We demonstrate that our approach achieves state-of-The-art performance on the ImageCLEF2011 dataset.Item What can pictures tell us about web pages? Improving document search using images(2013) Rodriguez-Vaamonde, Sergio; Torresani, Lorenzo; Fitzgibbon, Andrew; Tecnalia Research & InnovationTraditional Web search engines do not use the images in the HTML pages to find relevant documents for a given query. Instead, they typically operate by computing a measure of agreement between the keywords provided by the user and only the text portion of each page. In this paper we study whether the content of the pictures appearing in a Web page can be used to enrich the semantic description of an HTML document and consequently boost the performance of a keyword-based search engine. We present a Web-scalable system that exploits a pure text-based search engine to find an initial set of candidate documents for a given query. Then, the candidate set is reranked using semantic information extracted from the images contained in the pages. The resulting system retains the computational efficiency of traditional text-based search engines with only a small additional storage cost needed to encode the visual information. We test our approach on the TREC 2009 Million Query Track, where we show that our use of visual content yields improvement in accuracies for two distinct text-based search engines, including the system with the best reported performance on this benchmark.Item What can pictures tell us about web pages? Improving document search using images(2015-06-01) Rodriguez-Vaamonde, Sergio; Torresani, Lorenzo; Fitzgibbon, Andrew W.; Tecnalia Research & InnovationTraditional Web search engines do not use the images in the HTML pages to find relevant documents for a given query. Instead, they typically operate by computing a measure of agreement between the keywords provided by the user and only the text portion of each page. In this paper we study whether the content of the pictures appearing in a Web page can be used to enrich the semantic description of an HTML document and consequently boost the performance of a keyword-based search engine. We present a Web-scalable system that exploits a pure text-based search engine to find an initial set of candidate documents for a given query. Then, the candidate set is reranked using visual information extracted from the images contained in the pages. The resulting system retains the computational efficiency of traditional text-based search engines with only a small additional storage cost needed to encode the visual information. We test our approach on one of the TREC Million Query Track benchmarks where we show that the exploitation of visual content yields improvement in accuracies for two distinct text-based search engines, including the system with the best reported performance on this benchmark. We further validate our approach by collecting document relevance judgements on our search results using Amazon Mechanical Turk. The results of this experiment confirm the improvement in accuracy produced by our image-based reranker over a pure text-based system.