ZjXPSrSSKrSSKJrJrJr SSKJrJrJ r J r J r J r J r SSKJr SSKJrJr SSKJr S S KJrJrJrJrJrJrJrJr "S S \5r"S S\5r"SS\5r "SS\5r!"SS\5r""SS\5r#"SS\5r$"SS\5r%/SQr&g)zPyTorch CamemBERT model.N)BCEWithLogitsLossCrossEntropyLossMSELoss),BaseModelOutputWithPoolingAndCrossAttentions!CausalLMOutputWithCrossAttentionsMaskedLMOutputMultipleChoiceModelOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)Unpack)TransformersKwargsauto_docstring)can_return_tuple)RobertaForCausalLMRobertaForMaskedLMRobertaForMultipleChoiceRobertaForQuestionAnswering RobertaForSequenceClassificationRobertaForTokenClassification RobertaModelRobertaPreTrainedModelc\rSrSrSrSrg)CamembertPreTrainedModel,robertaN)__name__ __module__ __qualname____firstlineno__base_model_prefix__static_attributes__rڀ/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/camembert/modular_camembert.pyrr,s!r&rc\rSrSrSrg)CamembertModel0rN)r r!r"r#r%rr&r'r)r)0sr&r)c^\rSrSrSSS.rU4Sjr\\SS\RS-S\RS-S \RS-S \RS-S \RS-S \RS-S \RS-S\RS-S\ \ S\ \R\-4Sjj55rSrU=r$)CamembertForMaskedLM4)roberta.embeddings.word_embeddings.weight lm_head.biaszlm_head.decoder.weightzlm_head.decoder.biascF>[TU]U5 U?[USS9UlgNF)add_pooling_layersuper__init__ camembertr)rselfconfig __class__s r'r6CamembertForMaskedLM.__init__:#  N%fF r&N input_idsattention_masktoken_type_ids position_ids inputs_embedsencoder_hidden_statesencoder_attention_masklabelskwargsreturnc pUR"U4UUUUUUSS.U D6n U Sn URU 5n Sn UbaURU R5n[ 5nU"U R SUR R5UR S55n [U U U RU RS9$)a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` T)r?r@rArBrCrD return_dictrNlosslogits hidden_states attentions) rlm_headtodevicerviewr: vocab_sizer rNrO)r9r>r?r@rArBrCrDrErFoutputssequence_outputprediction_scoresmasked_lm_lossloss_fcts r'forwardCamembertForMaskedLM.forward@s:,,   ))%'"7#9    "!* LL9  YY0778F')H%&7&<&;?*.5 ##d*5 ))D05 ((4/ 5 &&- 5 ((4/ 5  %00475 !& 1 1D 85   4'5 +,5  u|| ~ -5 5 r&r,c8^\rSrSrU4Sjr\\SS\RS-S\RS-S\RS-S\RS-S\RS-S \RS-S \ \ S \ \R\-4S jj55rS rU=r$)"CamembertForSequenceClassificationzcF>[TU]U5 U?[USS9Ulgr2r4r8s r'r6+CamembertForSequenceClassification.__init__{r=r&Nr>r?r@rArBrErFrGc UR"U4UUUUSS.UD6nUSn URU 5n Sn UGbURU R5nURR cUR S:XaSURlOoUR S:aNUR[R:XdUR[R:XaSURlOSURlURR S:XaI[5n UR S:Xa&U "U R5UR55n OU "X5n OURR S:Xa=[5n U "U RS UR 5URS 55n O,URR S:Xa[5n U "X5n [!U U UR"UR$S 9$) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Tr?r@rArBrIrN regressionsingle_label_classificationmulti_label_classificationrJrK)r classifierrQrRr: problem_type num_labelsdtyper^longintrsqueezerrSrr rNrO r9r>r?r@rArBrErFrUrVrMrLrYs r'rZ*CamembertForSequenceClassification.forwards6,,  ))%'   "!*1  YYv}}-F{{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#F3D))-JJ+- B @&++b/R))-II,./'!//))   r&r\NNNNNN)r r!r"r#r6rrr^r_r`rrrarbr rZr%rcrds@r'rfrfzsG .237260426*.C ##d*C ))D0C ((4/ C &&- C ((4/ C   4'C +,C  u|| 7 7C C r&rfc8^\rSrSrU4Sjr\\SS\RS-S\RS-S\RS-S\RS-S\RS-S \RS-S \ \ S \ \R\-4S jj55rS rU=r$)CamembertForMultipleChoicecF>[TU]U5 U?[USS9Ulgr2r4r8s r'r6#CamembertForMultipleChoice.__init__r=r&Nr>r@r?rErArBrFrGc *UbURSOURSnUb!URSURS55OSn Ub!URSURS55OSn Ub!URSURS55OSn Ub!URSURS55OSn Ub1URSURS5URS55OSn UR"U 4U U U U SS.UD6nUSnUR U5nUR U5nURSU5nSnUb.UR UR5n[5nU"UU5n[UUURURS9$)aO input_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) token_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the multiple choice classification loss. Indices should be in `[0, ..., num_choices-1]` where `num_choices` is the size of the second dimension of the input tensors. (See `input_ids` above) position_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_choices, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. NrlrJT)rAr@r?rBrIrK) shaperSsizerdropoutrprQrRrr rNrO)r9r>r@r?rErArBrF num_choicesflat_input_idsflat_position_idsflat_token_type_idsflat_attention_maskflat_inputs_embedsrU pooled_outputrMreshaped_logitsrLrYs r'rZ"CamembertForMultipleChoice.forwardsV-6,Aiooa(}GZGZ[\G] CLCXINN2,>?^bLXLdL--b,2C2CB2GHjnR`Rln11"n6I6I"6MNrvR`Rln11"n6I6I"6MNrv(   r=#5#5b#9=;M;Mb;Q R  ,,  *..,      ]3 / ++b+6  YY556F')HOV4D("!//))   r&r\ry)r r!r"r#r6rrr^r_r`rrrarbr rZr%rcrds@r'r{r{sG .22637*.0426P ##d*P ((4/P ))D0 P   4' P &&- P ((4/P +,P  u|| 8 8P P r&r{c8^\rSrSrU4Sjr\\SS\RS-S\RS-S\RS-S\RS-S\RS-S \RS-S \ \ S \ \R\-4S jj55rS rU=r$)CamembertForTokenClassificationi%cF>[TU]U5 U?[USS9Ulgr2r4r8s r'r6(CamembertForTokenClassification.__init__&r=r&Nr>r?r@rArBrErFrGc zUR"U4UUUUSS.UD6nUSn URU 5n URU 5n Sn UbWURU R5n[ 5n U "U R SUR5UR S55n [U U URURS9$)a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. TrkrNrJrK) rrrprQrRrrSrrr rNrOrws r'rZ'CamembertForTokenClassification.forward,s2,,  ))%'   "!*,,71  YYv}}-F')HFKKDOO[TU]U5 U?[USS9Ulgr2r4r8s r'r6&CamembertForQuestionAnswering.__init__dr=r&Nr>r?r@rArBstart_positions end_positionsrFrGc UR"U4UUUUSS.UD6n U Sn URU 5n U RSSS9upU RS5R 5n U RS5R 5n SnUbUb[ UR 55S:aURS5n[ UR 55S:aURS5nU R S5nURSU5nURSU5n[US9nU"X5nU"X5nUU-S - n[UU U U RU RS 9$) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) TrkrrlrJ)dimN) ignore_indexr)rL start_logits end_logitsrNrO) r qa_outputssplitrv contiguouslenrclamprr rNrO)r9r>r?r@rArBrrrFrUrVrMrr total_loss ignored_indexrY start_lossend_losss r'rZ%CamembertForQuestionAnswering.forwardjsx0,,  ))%'   "!*1#)<<r<#: #++B/::< ''+668   &=+D?'')*Q."1"9"9""==%%'(1, - 5 5b 9 (--a0M-33A}EO)//=AM']CH!,@J :H$x/14J+%!!//))   r&r\)NNNNNNN)r r!r"r#r6rrr^r_r`rrrarbr rZr%rcrds@r'rrcsG .2372604263715> ##d*> ))D0> ((4/ > &&- > ((4/ > ))D0> ''$.> +,>  u|| ; ;> > r&rc^\rSrSrSSS.rU4Sjr\\SS\RS-S\RS-S \RS-S \RS-S \RS-S \RS-S \RS-S\RS-S\ \ \RS-S\ S-S\ \R-S\\S\ \R\-4Sjj55rSrU=r$)CamembertForCausalLMir.r/r0cF>[TU]U5 U?[USS9Ulgr2r4r8s r'r6CamembertForCausalLM.__init__r=r&Nr>r?r@rArBrCrDrEpast_key_values use_cachelogits_to_keeprFrGc UbSn UR"U4UUUUUUU U SS. U D6n U Rn[U [5(a [ U *S5OU nUR USS2USS245nSnUb)UR "SUXRRS.U D6n[UUU RU RU RU RS9$)a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` Example: ```python >>> from transformers import AutoTokenizer, CamembertForCausalLM, AutoConfig >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("almanach/camembert-base") >>> config = AutoConfig.from_pretrained("almanach/camembert-base") >>> config.is_decoder = True >>> model = CamembertForCausalLM.from_pretrained("almanach/camembert-base", config=config) >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> prediction_logits = outputs.logits ```NFT) r?r@rArBrCrDrrrI)rMrErT)rLrMrrNrOcross_attentionsr)rlast_hidden_state isinstanceruslicerP loss_functionr:rTrrrNrOr)r9r>r?r@rArBrCrDrErrrrFrUrN slice_indicesrMrLs r'rZCamembertForCausalLM.forwards`  I@D  A ))%'"7#9+ A  A  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVF{{OeOepiopD0#33!//))$55   r&r\) NNNNNNNNNNr)r r!r"r#r]r6rrr^r_r`raboolrurbrrrrZr%rcrds@r'rrst"M . G .237260426:>;?*.BF!%-.O ##d*O ))D0O ((4/ O &&- O ((4/ O  %0047O !& 1 1D 8O   4'O uU%6%6784?O $;O ell*O +,O  u|| @ @O O r&r)rr,r{rrfrr)r)'__doc__r^torch.nnrrrmodeling_outputsrrr r r r r processing_utilsrutilsrr utils.genericrroberta.modeling_robertarrrrrrrrrr)r,rfr{rrr__all__rr&r'rs AA'7-   "5" \ C -C LL )IL ^Y !9Y x; &C; |G $?G T] -] @ r&