• Publications
    • Conference Papers
    • Workshop Papers
    • Journal Papers
    • Publicity
    • Books
    • Theses
    • Submitted
  • Professional Activities
  • Teaching
  • About
  • Contact

Edgar Meij

semantic search research ッ

  • Publications
    • Conference Papers
    • Workshop Papers
    • Journal Papers
    • Publicity
    • Books
    • Theses
    • Submitted
  • Professional Activities
  • Teaching
  • About
  • Contact
TREC

TREC 2012 summary

09/11/2012 Blog No Comments

In the 21st Text REtrieval Conference (TREC 2012), seven tracks ran: KBA, Contextual suggestion, Session, Web, Medical, Crowdsourcing, and Microblog. Of these, Microblog attracted the largest number of participating groups (40) closely followed by Medical (24). UvA mainly participated in KBA (Knowledge Base Acceleration) and was one of 11 participating groups. The KBA task is a typical cumulative citation recommendation task, where a stream of documents is filtered for relevance. In this case, relevance is determined using entities, i.e., the use-case is a Wikipedia editor with an interest in a certain Wikipedia article (entity) and she needs to be notified of “interesting” documents, that are “central” to the entity.

In our participation we evaluated a previously proposed approach on the KBA test collection and extended it to accommodate the temporal, evolving nature of the document collection. Our official runs contained a bug, but the repaired version obtains encouraging performance (and can be found online). There was some degree of variability between the KBA approaches, although most of them used entity “representations” in some way or other. CWI, for instance, used the Google anchor-concept dump, UDel used outlinks inside the Wikipedia article, and UIUC (Miles Efron) used the article’s edit history. UMass (Jeff Dalton) included the same entity name variants as us, including titles, redirects, and anchors. UMass also explicitly addressed the connection between TREC-KBA and TAC-KBP, hopefully resulting in the two tracks moving together (KBx? KBY?). In any case, I’m looking forward to next year’s KBA, where most of these approaches will (hopefully) be combined to further improve performance.

As to TREC 2013, there will be quite some changes. TREC Medical stops, mainly due to issues with the medical records document collection. There will be two new tracks: Temporal summarization (Fernando Diaz) and Federated web search (Djoerd Hiemstra). Especially the first one seems interesting from a KBA point of view. Furthermore, TREC-TempSum will use the same document collection as TREC-KBA in 2013 (hopefully also including tweets and Facebook updates), fostering further possible integration between the tracks. While still unclear, TREC-Microblog (called TREC-RealTime next year) is contemplating using this collection as well. Speaking of new collections, TREC-Web 2013 also features a new collection, ClueWeb12 (as well as a new task: risk-sensitive retrieval). And it seems ClueWeb12 might also be used by the contextual suggestion and crowdsourcing tracks in 2013.

All in all, it was an exciting edition of TREC with lots of interesting discussions. Food for thought, not only for next year’s TREC but also for the upcoming SIGIR deadline :).

2013-crowdsorsingoverview-of-the-trec-2012-medical-records-tracktemporal-summarization-track-2013temporal-summarization-trec-2013track-medical-record-textTRECTREC KBATREC MedicalTREC MicroblogTREC Sessionstrec-2012-medical-records-tracktrec-2012-result-contextualtrec-2013trec-2013-crowdsourcingtrec-2013-temporaltrec-2013-temporal-summarizationtrec-2013-temporal-summarization-tracktrec-medical-records-track-2012trec-retrievaltrec2012

The University of Amsterdam at TREC 2012

Real-Time Rank Aggregation for Microblog Search

Leave a Reply Cancel reply

Time limit is exhausted. Please reload CAPTCHA.

Edgar Meij logo
Welcome!

This is the website of Edgar Meij. I lead several groups of researchers and engineers at Bloomberg working on knowledge graphs, question answering, information retrieval, machine learning, and more…

Search
Tweets by @edgarmeij
Tags
AIDA Artificial Intelligence CLEF content DBpedia edgar-meij entity-linking-and-retrieval entity-linking-and-retrieval-tutorial entity-linking-tutorial Entity finding Entity linking Information retrieval Knowledge base population Knowledge Graph Language modeling Linking Open Data LOD logo-penerbit-buku-internasional Machine learning meij MeSH Microblogs penerbit-buku-internasional personalized-time-aware-tweets-summarization Query log analysis Query modeling Relevance modeling Semanticizing Semantic linking Semantic query analysis Semantic search Teaching Text mining TREC Blog TREC Genomics TREC KBA TREC Microblog TREC Relevance Feedback TREC Sessions Tutorial Twitter Web services Wikipedia Workflows Workshop
Proudly powered by WordPress | Theme: Doo by ThemeVS.