capreolus.extractor.common¶
Module Contents¶
Classes¶
Prepare and parse TF training feature that contain multiple passage per query. |
|
Prepare and parse TF training feature that contain single passage per query. |
Functions¶
|
|
|
|
|
Attributes¶
- class capreolus.extractor.common.MultipleTrainingPassagesMixin[source]¶
Prepare and parse TF training feature that contain multiple passage per query. That is, the “pos_bert_input” features prepared by extractor’s id2vec() function should have 3 dimension
- create_tf_train_feature(sample)[source]¶
Returns a set of features from a doc. Of the num_passages passages that are present in a document, we use only a subset of it. params: sample - A dict where each entry has the shape [batch_size, num_passages, maxseqlen] Returns a list of features. Each feature is a dict, and each value in the dict has the shape [batch_size, maxseqlen]. Yes, the output shape is different to the input shape because we sample from the passages.
- class capreolus.extractor.common.SingleTrainingPassagesMixin[source]¶
Prepare and parse TF training feature that contain single passage per query. That is, the “pos_bert_input” features prepared by extractor’s id2vec() function should have 2 dimension
- create_tf_train_feature(sample)[source]¶
Returns a set of features from a doc. Of the num_passages passages that are present in a document, we use only a subset of it. params: sample - A dict where each entry has the shape [batch_size, num_passages, maxseqlen]
Returns a list of features. Each feature is a dict, and each value in the dict has the shape [batch_size, maxseqlen]. Yes, the output shape is different to the input shape because we sample from the passages.