We describe the par­tic­i­pa­tion of the Uni­ver­sity of Amsterdam’s ILPS group in the Ses­sion track at TREC 2011.

The stream of inter­ac­tions cre­ated by a user engag­ing with a search sys­tem con­tains a wealth of infor­ma­tion. For retrieval pur­poses, pre­vi­ous inter­ac­tions can help inform us about a user’s cur­rent infor­ma­tion need. Build­ing on this intu­ition, our con­tri­bu­tion to this TREC year’s ses­sion track focuses on ses­sion mod­el­ing and learn­ing to rank using ses­sion infor­ma­tion. In this paper, we present and com­pare three com­ple­men­tary strate­gies that we designed for improv­ing retrieval for a cur­rent query using pre­vi­ous queries and clicked results: prob­a­bilis­tic ses­sion mod­el­ing, seman­tic query mod­el­ing, and implicit feedback.

In our exper­i­ments we exam­ined three com­ple­men­tary strate­gies for improv­ing retrieval for a cur­rent query. Our first strat­egy, based on prob­a­bilis­tic ses­sion mod­el­ing, was the best per­form­ing strategy.

Our sec­ond strat­egy, based on seman­tic query mod­el­ing, did less well than we expected, likely due to topic drift from exces­sively aggres­sive query expan­sion. We expect that per­for­mance of this strat­egy would improve by lim­it­ing the num­ber of terms and/or improv­ing the prob­a­bil­ity estimates.

With respect to our third strat­egy, based on learn­ing from feed­back, we found that learn­ing weights for lin­ear weighted com­bi­na­tions of fea­tures from an exter­nal col­lec­tion can be ben­e­fi­cial, if char­ac­ter­is­tics of the col­lec­tion are sim­i­lar to the cur­rent data. Feed­back avail­able in the form of user clicks appeared to be less ben­e­fi­cial. Our run learn­ing from implicit feed­back did per­form sub­stan­tially lower than a run where weights were learned from an exter­nal col­lec­tion with explicit feed­back using the same learn­ing algo­rithm and set of features.

  • [PDF] B. Huurnink, R. Berend­sen, K. Hof­mann, E. Meij, and M. de Rijke, “The Uni­ver­sity of Ams­ter­dam at the TREC 2011 Ses­sion Track,” in Pro­ceed­ings of The Twen­ti­eth Text REtrieval Con­fer­ence, TREC 2011, 2011.
    [Bib­tex]
    @inproceedings{TREC:2011:huurnink,
  Author = {Huurnink, Bouke and Berendsen, Richard and Hofmann, Katja and Meij, Edgar and de Rijke, Maarten},
  Booktitle = {Proceedings of The Twentieth Text REtrieval Conference, TREC 2011},
  Date-Added = {2011-10-22 12:22:18 +0200},
  Date-Modified = {2012-02-12 14:02:18 +0100},
  Editor = {Ellen M. Voorhees and Lori Buckland},
  Publisher = {National Institute of Standards and Technology ({NIST})},
  Title = {The University of Amsterdam at the {TREC} 2011 Session Track},
  Year = {2011}}