||Base class for Extractor modules. The purpose of an Extractor is to convert queries and documents to a representation suitable for use with a
BertText(config=None, provide=None, share_dependency_objects=False, build=True)[source]¶
Base class for Extractor modules. The purpose of an Extractor is to convert queries and documents to a representation suitable for use with a
- Modules should provide:
id2vec(qid, posid, negid=None)method that converts the given query and document ids to an appropriate representation
sample - output from self.id2vec() return - a tensorflow feature
id2vec(self, qid, posid, negid=None)[source]¶
Creates a feature from the (qid, docid) pair. If negdocid is supplied, that’s also included in the feature (needed for training with pairwise hinge loss) Label is a vector of shape [num_classes], and is supplied only when using pointwise training (i.e cross entropy) When using pointwise samples, negdocid is None, and label is either [0, 1] or [1, 0] depending on whether the document represented by posdocid is relevant or irrelevant respectively.