capreolus.utils.common

Module Contents

Classes

Anserini

OrderedDefaultDict

dict() -> new empty dictionary

Functions

download_file(url, outfn, expected_hash=None)

Download url to the file outfn. If expected_hash is provided, use it to both verify the file was downloaded

hash_file(fn)

Compute a SHA-256 hash for the file fn and return a hexdigest of the hash

padlist(list_to_pad, padlen, pad_token=0)

Pads a list with pad_token

plot_loss(history, outfn, interactive=False)

plot_metrics(metrics, outfn, interactive=False, show={'map', 'P_20', 'ndcg_cut_20'})

remove_newline(txt)

get_udel_query_expander()

Attributes

logger

capreolus.utils.common.logger[source]
class capreolus.utils.common.Anserini[source]
classmethod get_fat_jar(cls)[source]
classmethod filter_and_log_anserini_output(cls, line, logger)[source]

Ignore DEBUG lines and require other lines pass our logging level

capreolus.utils.common.download_file(url, outfn, expected_hash=None)[source]

Download url to the file outfn. If expected_hash is provided, use it to both verify the file was downloaded correctly, and to avoid re-downloading an existing file with a matching hash.

capreolus.utils.common.hash_file(fn)[source]

Compute a SHA-256 hash for the file fn and return a hexdigest of the hash

capreolus.utils.common.padlist(list_to_pad, padlen, pad_token=0)[source]

Pads a list with pad_token :param list_to_pad: a list of str / int / float, each entry represents the token / token id :param padlen: int, max length of the expected list, list_to_pad would either be truncated or padded to this length :param pad_token: str / int, representing the padding token / token id

Returns: padded list

capreolus.utils.common.plot_loss(history, outfn, interactive=False)[source]
capreolus.utils.common.plot_metrics(metrics, outfn, interactive=False, show={'map', 'P_20', 'ndcg_cut_20'})[source]
capreolus.utils.common.remove_newline(txt)[source]
capreolus.utils.common.get_udel_query_expander()[source]
class capreolus.utils.common.OrderedDefaultDict[source]

Bases: collections.OrderedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)