capreolus.utils.common

Module Contents

Classes

Anserini
OrderedDefaultDict dict() -> new empty dictionary

Functions

download_file(url, outfn, expected_hash=None) Download url to the file outfn. If expected_hash is provided, use it to both verify the file was downloaded
hash_file(fn) Compute a SHA-256 hash for the file fn and return a hexdigest of the hash
padlist(list_to_pad, padlen, pad_token=0) Pads a list with pad_token
plot_loss(history, outfn, interactive=False)
plot_metrics(metrics, outfn, interactive=False, show={‘map’, ‘P_20’, ‘ndcg_cut_20’})
remove_newline(txt)
get_udel_query_expander()
capreolus.utils.common.logger[source]
class capreolus.utils.common.Anserini[source]
classmethod get_fat_jar(cls)[source]
classmethod filter_and_log_anserini_output(cls, line, logger)[source]

Ignore DEBUG lines and require other lines pass our logging level

capreolus.utils.common.download_file(url, outfn, expected_hash=None)[source]

Download url to the file outfn. If expected_hash is provided, use it to both verify the file was downloaded correctly, and to avoid re-downloading an existing file with a matching hash.

capreolus.utils.common.hash_file(fn)[source]

Compute a SHA-256 hash for the file fn and return a hexdigest of the hash

capreolus.utils.common.padlist(list_to_pad, padlen, pad_token=0)[source]

Pads a list with pad_token :param list_to_pad: a list of str / int / float, each entry represents the token / token id :param padlen: int, max length of the expected list, list_to_pad would either be truncated or padded to this length :param pad_token: str / int, representing the padding token / token id

Returns: padded list

capreolus.utils.common.plot_loss(history, outfn, interactive=False)[source]
capreolus.utils.common.plot_metrics(metrics, outfn, interactive=False, show={'map', 'P_20', 'ndcg_cut_20'})[source]
capreolus.utils.common.remove_newline(txt)[source]
capreolus.utils.common.get_udel_query_expander()[source]
class capreolus.utils.common.OrderedDefaultDict[source]

Bases: collections.OrderedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs
dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
in the keyword argument list. For example: dict(one=1, two=2)