capreolus.sampler

Package Contents

Classes

TrainDataset(qid_docid_to_rank, qrels, extractor) Samples training data. Intended to be used with a pytorch DataLoader
PredDataset(qid_docid_to_rank, extractor) Creates a Dataset for evaluation (test) data to be used with a pytorch DataLoader
capreolus.sampler.logger[source]
class capreolus.sampler.TrainDataset(qid_docid_to_rank, qrels, extractor)[source]

Bases: torch.utils.data.IterableDataset

Samples training data. Intended to be used with a pytorch DataLoader

get_hash(self)[source]
get_total_samples(self)[source]
generator_func(self)[source]
class capreolus.sampler.PredDataset(qid_docid_to_rank, extractor)[source]

Bases: torch.utils.data.IterableDataset

Creates a Dataset for evaluation (test) data to be used with a pytorch DataLoader

get_hash(self)[source]
get_qid_docid_pairs(self)[source]

Returns a generator for the (qid, docid) pairs. Useful if you want to sequentially access the pred pairs without extracting the actual content