capreolus.benchmark.robust04

Module Contents

Classes

Robust04

Robust04 benchmark using the title folds from Huston and Croft. [1] Each of these is used as the test set.

Robust04Yang19

Robust04 benchmark using the folds from Yang et al. [1]

Robust04Yang19Desc

Robust04 benchmark using the folds from Yang et al. [1]

capreolus.benchmark.robust04.PACKAGE_PATH[source]
class capreolus.benchmark.robust04.Robust04(config=None, provide=None, share_dependency_objects=False, build=True)[source]

Bases: capreolus.benchmark.Benchmark

Robust04 benchmark using the title folds from Huston and Croft. [1] Each of these is used as the test set. Given the remaining four folds, we split them into the same train and dev sets used in recent work. [2]

[1] Samuel Huston and W. Bruce Croft. 2014. Parameters learned in the comparison of retrieval models using term dependencies. Technical Report.

[2] Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli Goharian. 2019. CEDR: Contextualized Embeddings for Document Ranking. SIGIR 2019.

module_name = robust04[source]
dependencies[source]
qrel_file[source]
topic_file[source]
fold_file[source]
query_type = title[source]
class capreolus.benchmark.robust04.Robust04Yang19(config=None, provide=None, share_dependency_objects=False, build=True)[source]

Bases: capreolus.benchmark.Benchmark

Robust04 benchmark using the folds from Yang et al. [1]

[1] Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. 2019. Critically Examining the “Neural Hype”: Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. SIGIR 2019.

module_name = robust04.yang19[source]
dependencies[source]
qrel_file[source]
topic_file[source]
fold_file[source]
query_type = title[source]
class capreolus.benchmark.robust04.Robust04Yang19Desc(config=None, provide=None, share_dependency_objects=False, build=True)[source]

Bases: capreolus.benchmark.robust04.Robust04Yang19, capreolus.benchmark.Benchmark

Robust04 benchmark using the folds from Yang et al. [1]

[1] Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. 2019. Critically Examining the “Neural Hype”: Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. SIGIR 2019.

module_name = robust04.yang19.desc[source]
query_type = desc[source]