ZjLU8SrSSKJr SSKrSSKJrJr SSKJr SSKJ r SSK J r J r J r S S KJr S S KJr \ R$"\5r\ "S S9\"SS\ 555r\ "S S9\"SS\ 555r\ "S S9\"SS\ 555r\ "SS\ 55r"SS\5r"SS\5r"SS\5r"SS\5r"SS \5r\ "S!S9"S"S#\55r\ "S$S9"S%S&\55r\ "S'S9"S(S)\55r /S*Qr!g)+z5PyTorch DPR model for Open Domain Question Answering.) dataclassN)Tensornn)BaseModelOutputWithPooling)PreTrainedModel) ModelOutputauto_docstringlogging) BertModel) DPRConfigz6 Class for outputs of [`DPRQuestionEncoder`]. ) custom_introc\rSrSr%Sr\R \S'Sr\ \R S4S-\S'Sr \ \R S4S-\S'Sr g) DPRContextEncoderOutput(a pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the context representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed contexts for nearest neighbors queries with questions embeddings. pooler_outputN. hidden_states attentions __name__ __module__ __qualname____firstlineno____doc__torch FloatTensor__annotations__rtupler__static_attributes__ru/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/dpr/modeling_dpr.pyrr(S$$$:>M5**C/047>7;Je'',-4;r#rc\rSrSr%Sr\R \S'Sr\ \R S4S-\S'Sr \ \R S4S-\S'Sr g) DPRQuestionEncoderOutput;a pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the question representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed questions for nearest neighbors queries with context embeddings. rN.rrrrrr#r$r'r';r%r#r'c\rSrSr%Sr\R \S'Sr\R S-\S'Sr \R S-\S'Sr \ \R S4S-\S'Sr \ \R S4S-\S 'S r g) DPRReaderOutputNa start_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the start index of the span for each passage. end_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the end index of the span for each passage. relevance_logits (`torch.FloatTensor` of shape `(n_passages, )`): Outputs of the QA classifier of the DPRReader that corresponds to the scores of each passage to answer the question, compared to all the other passages. start_logitsN end_logitsrelevance_logits.rrr)rrrrrrrr r-r.rr!rr"rr#r$r*r*Ns~###+/J!!D(/15e''$.5:>M5**C/047>7;Je'',-4;r#r*c\rSrSrSrSrg)DPRPreTrainedModelfTrN)rrrr_supports_sdpar"rr#r$r0r0fsNr#r0c^\rSrSrSrS\4U4SjjrSS\S\S-S\S-S \S-S \S \S \S \ \ \S4-4Sjjr \ S \ 4Sj5rSrU=r$) DPREncoderk bert_modelconfigc>[TU]U5 [USS9UlURRR S::a [ S5eURUlURS:aD[R"URRR UR5Ul UR5 g)NF)add_pooling_layerrz!Encoder hidden_size can't be zero) super__init__r r6r7 hidden_size ValueErrorprojection_dimrLinear encode_proj post_initselfr7 __class__s r$r;DPREncoder.__init__ns  #FeD ?? ! ! - - 2@A A$33    "!yy)?)?)K)KVMbMbcD  r#N input_idsattention_masktoken_type_ids inputs_embedsoutput_attentionsoutput_hidden_states return_dictreturn.c URUUUUUUUS9n U Sn U SS2SSS24n URS:aURU 5n U(d X4U SS-$[U U U RU R S9$)NrFrGrHrIrJrKrLrr )last_hidden_staterrr)r6r>r@rrr) rCrFrGrHrIrJrKrLkwargsoutputssequence_output pooled_outputs r$forwardDPREncoder.forwardys//))'/!5#" "!*'1a0    " ,,];M#3gabkA A)-'!//))   r#cURS:aURR$URRR $)Nr)r>r@ out_featuresr6r7r<)rCs r$embeddings_sizeDPREncoder.embeddings_sizes8    "##00 0%%111r#)r6r@r>)NNNFFF)rrrrbase_model_prefixrr;rboolrr!rUpropertyintrYr" __classcell__rDs@r$r4r4ks$ y )-(,'+"'%*!" "  "  " } "  " #" "  $eFCK&8 8" H222r#r4c|^\rSrSrSrS\4U4SjjrSS\S\S\S-S \S \S \S \ \ \S 4-4Sjjr Sr U=r $)DPRSpanPredictorencoderr7c$>[TU]U5 [U5Ul[R "URR S5Ul[R "URR S5UlUR5 g)Nr r) r:r;r4rdrr?rY qa_outputs qa_classifierrArBs r$r;DPRSpanPredictor.__init__s_  !&) ))DLL$@$@!DYYt||'C'CQG r#NrFrGrIrJrKrLrM.c FUbUR5OUR5SSupURUUUUUUS9n U Sn URU 5n U RSSS9upU R S5R 5n UR S5R 5nUR U SS2SSS245nU RX5n URX5nURU5nU(d XU4U SS-$[U UUU RU RS9$)Nr )rGrIrJrKrLrr)dim)r,r-r.rr) sizerdrfsplitsqueeze contiguousrgviewr*rr)rCrFrGrIrJrKrLrQ n_passagessequence_lengthrRrSlogitsr,r-r.s r$rUDPRSpanPredictor.forwardsD;D:Oinn&6UbUgUgUijlklUm# ,, )'/!5#  "!*1#)<<r<#: #++B/::< ''+668 --oaAg.FG$((E __ZA +00< .>?'!"+M M%!-!//))   r#)rdrgrf)NFFF)rrrrr[rr;rr\r*r!rUr"r_r`s@r$rbrbs!y(,"'%*!, , , } ,  , # , ,  5- -, , r#rbc(\rSrSr%Sr\\S'SrSrg)DPRPretrainedContextEncoderz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. r7 ctx_encoderrN rrrrrrr r[r"rr#r$rvrvs %r#rvc(\rSrSr%Sr\\S'SrSrg)DPRPretrainedQuestionEncoderrxr7question_encoderrNrzrr#r$r|r|s *r#r|c(\rSrSr%Sr\\S'SrSrg)DPRPretrainedReaderrxr7span_predictorrNrzrr#r$rrs (r#rzf The bare DPRContextEncoder transformer outputting pooler outputs as context representations. c^\rSrSrS\4U4Sjjr\SS\S-S\S-S\S-S\S-S \S-S \S-S \S-S \ \ \S 4-4Sjj5r Sr U=r $)DPRContextEncoderir7cp>[TU]U5 Xl[U5UlUR 5 gN)r:r;r7r4ryrArBs r$r;DPRContextEncoder.__init__ s,   %f- r#NrFrGrHrIrJrKrLrM.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUbUR 5n O"UbUR 5SSn O [ S5eUb UR O UR n Uc/Uc[R"XS9OXRR:gnUc$[R"U [RU S9nURUUUUUUUS9n U(dU SS$[U RU RU R S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRContextEncoder, DPRContextEncoderTokenizer >>> tokenizer = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> model = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ```NDYou cannot specify both input_ids and inputs_embeds at the same timerj5You have to specify either input_ids or inputs_embedsdevicedtyperrOrrrr)r7rJrKrLr=rlrrones pad_token_idzeroslongryrrrr rCrFrGrHrIrJrKrLrQ input_shaperrRs r$rUDPRContextEncoder.forwardsld2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN""))'/!5## 12; &!//w?T?Tahasas  r#)r7ryNNNNNNN)rrrrrr;r rr\rr!rUr"r_r`s@r$rrs y$((,(,'+)-,0#'Y D=Y  Y  Y } Y  $; Y #TkY D[Y  !5#5 5Y Y r#rzh The bare DPRQuestionEncoder transformer outputting pooler outputs as question representations. c^\rSrSrS\4U4Sjjr\SS\S-S\S-S\S-S\S-S \S-S \S-S \S-S \ \ \S 4-4Sjj5r Sr U=r $)DPRQuestionEncoderipr7cp>[TU]U5 Xl[U5UlUR 5 gr)r:r;r7r4r~rArBs r$r;DPRQuestionEncoder.__init__vs,   *6 2 r#NrFrGrHrIrJrKrLrM.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5n O"UbUR 5SSn O [ S5eUb URO URn Uc/Uc[R"XS9OXRR:gnUc$[R"U [RU S9nURUUUUUUUS9n U(dU SS$[U RU R U R"S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRQuestionEncoder, DPRQuestionEncoderTokenizer >>> tokenizer = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> model = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ``` NrrjrrrrOrr)r7rJrKrLr=%warn_if_padding_and_no_attention_maskrlrrrrrrr~r'rrrrs r$rUDPRQuestionEncoder.forward}szd2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN''))'/!5#( 12; '!//w?T?Tahasas  r#)r7r~r)rrrrrr;r rr\r'r!rUr"r_r`s@r$rrps y$((,(,'+)-,0#'Z D=Z  Z  Z } Z  $; Z #TkZ D[Z  "E&#+$6 6Z Z r#rzE The bare DPRReader transformer outputting span predictions. c^\rSrSrS\4U4Sjjr\SS\S-S\S-S\S-S\S-S \S-S \S-S \ \ \S 4-4S jj5r Sr U=r $) DPRReaderir7cp>[TU]U5 Xl[U5UlUR 5 gr)r:r;r7rbrrArBs r$r;DPRReader.__init__s,   .v6 r#NrFrGrIrJrKrLrM.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5nO"UbUR 5SSnO [ S5eUb URO URn Uc[R"XS9nURUUUUUUS9$)a input_ids (`tuple[torch.LongTensor]` of shapes `(n_passages, sequence_length)`): Indices of input sequence tokens in the vocabulary. It has to be a sequence triplet with 1) the question and 2) the passages titles and 3) the passages texts To match pretraining, DPR `input_ids` sequence should be formatted with [CLS] and [SEP] with the format: `[CLS] [SEP] [SEP] ` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`DPRReaderTokenizer`]. See this class documentation for more details. [What are input IDs?](../glossary#input-ids) inputs_embeds (`torch.FloatTensor` of shape `(n_passages, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. Examples: ```python >>> from transformers import DPRReader, DPRReaderTokenizer >>> tokenizer = DPRReaderTokenizer.from_pretrained("facebook/dpr-reader-single-nq-base") >>> model = DPRReader.from_pretrained("facebook/dpr-reader-single-nq-base") >>> encoded_inputs = tokenizer( ... questions=["What is love ?"], ... titles=["Haddaway"], ... texts=["'What Is Love' is a song recorded by the artist Haddaway"], ... return_tensors="pt", ... ) >>> outputs = model(**encoded_inputs) >>> start_logits = outputs.start_logits >>> end_logits = outputs.end_logits >>> relevance_logits = outputs.relevance_logits ``` Nrrjrr)rIrJrKrL) r7rJrKrLr=rrlrrrr) rCrFrGrIrJrKrLrQrrs r$rUDPRReader.forwardsb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ CN""  '/!5# #  r#)r7r)NNNNNN)rrrrrr;r rr\r*r!rUr"r_r`s@r$rrs y$((,'+)-,0#'L D=L  L } L  $; L #Tk L D[L  5- -L L r#r)rrvr0r|rrr)"r dataclassesrrrrmodeling_outputsrmodeling_utilsrutilsr r r bert.modeling_bertr configuration_dprr get_loggerrloggerrr'r*r0r4rbrvr|rrrr__all__rr#r$rs<! :- +(   H %