ZjP`%SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK J r SSKJ r SSK J r Jr SSKJr SSKJrJr SSKJrJr SSKJr SS KJr SS KJrJrJrJrJr SS K J!r! SSK"r"SS K#J$r$J%r%J&r& SS K'J(r( SSK)J*r* SSK+J,r- SSK+J.r/ SSK"J0r0J1r1 SSK2J3r3 SSK4J5r5 SSK6J7r8 SSK9J:r: SSK;Jr>J?r?J@r@JArA SSKBJCrC SSKDJErE SSKFJGrGJHrH SSKIJJrJJKrKJLrLJMrMJNrN SSKOJPrPJQrQJRrRJSrSJTrTJUrUJVrV SSKWJXrX SSKYJZrZ SS K[J\r\ SS!K]J^r^ SS"K_J`r` SS#KaJbrb SS$KcJdrdJere SS%KfJgrg SS&KhJiri SS'KjJkrk SS(KlJmrm SS)KnJoroJprpJqrqJrrrJsrsJtrtJuru SS*KvJwrw SS+KxJyryJzrzJ{r{J|r| SS,K}J~r~ SS-KJrJr SS.KJr SS/KJr SS0KJr SS1KJr SS2KJr SS3KJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJr SS4KJrJrJr SS5KJrJrJr SS6KJrJrJrJrJr SS7KJrJr SS8KJrJr SS9KJr \"5(a SS:KJr SS;KJr \(aSS5:rOS?r\GR"\5r\GRGRS@SA5GR5r\GRGRSBSA5GR5r\"SCSDSE9rS?qS?q\"SFSG9"SHSI55rSJ\4SKjrSJ\4SLjrSMr\SN5r\SO5r\SSP\"GRSQ\S-4SRjj5rSSrSTr\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GR\"GRSU.rSVSWSJ\4SXjrSSY\\GR-SZ\\"GR-S[\S\\S-SJ\\\"R`44 S]jjrS^\"R`SJ\4S_jrS`\1GRSJ\\4SajrSb\\\Sc\\\"R`4SJ\\\\\\44SdjrSb\\\Sc\\\"R`4SJ\\\\\\\44SejrSc\\\"R`4SfSDSJ\\\"R`44SgjrSfSDSh\S^\"R`4SijrSSj\Sk\S-SJ\4SljjrSSm\\GR-S-Sk\S-Sn\S-So\S-Sp\S-Sq\Sr\S-Ss\S-St\S-SJ\\\S-\S-44SujjGrSSP\\"GR-\-S-Sv\\S-Sw\:Sx\S-Sc\S-S[\Sy\S-SJ\\:\"GR44SzjjGr"S{S|5Gr"S}S~5Gr"SSD\1GRG\G\\\J5Gr\"G\GR 5G\GlG\GR GR b;G\GR GR GRSfSSS9G\GR Gl\SSfG\S\SJG\4Sjj5Gr\SSf\1GRS\SJ\1GR4Sjj5GrSSf\1GRS\SJ\1GR4SjjGrS\\-\"GR-SJ\4SjGr SSfG\S\Sy\S-4SjjGr SfG\S\Sy\S-4SjGr "SS\5Gr G\ "5Gr G\ G\S'"SSG\5Grg)N)abstractmethod) defaultdict)CallableIterator)contextmanager) dataclassfield)partialwraps)cycle)Thread) TYPE_CHECKINGAnyTypeVarget_type_hintsoverload) is_zipfile) create_repois_offline_mode"split_torch_state_dict_into_shards)version) safe_open)load) save_file)Tensornn) constraints) checkpoint)initialization)PreTrainedConfig)get_model_conversion_mapping)WeightConverterWeightRenaming$convert_and_load_state_dict_in_modelrevert_weight_conversion)DistributedConfig)custom_object_save) CompileConfigGenerationConfig)PeftAdapterMixindeepspeed_config hub_kernelsis_deepspeed_zero3_enabledis_fsdp_enabled)_get_device_mapaccelerate_disk_offloadaccelerate_dispatchcheck_and_set_device_mapexpand_device_map get_deviceload_offloaded_parameter)!_load_state_dict_into_zero3_model)eager_paged_attention_forward)ALL_FP8_EXPERTS_FUNCTIONS)flash_attention_forward)paged_attention_forward)flex_attention_forward)allow_all_hub_kernels is_kernel)ALL_EXPERTS_FUNCTIONS)maybe_load_adapters)sdpa_attention_forward)sdpa_attention_paged_forward)ALL_PARALLEL_STYLES_get_parameter_tp_plandistribute_modelgather_state_dict_for_saveinitialize_tensor_parallelismshard_and_distribute_moduleverify_tp_plan) LOSS_MAPPING)$FLASH_ATTENTION_COMPATIBILITY_MATRIXFLASH_ATTN_KERNEL_FALLBACKlazy_import_flash_attention!lazy_import_paged_flash_attention)ROPE_INIT_FUNCTIONS) apply_patchespatch_output_recorders)id_tensor_storage) HfQuantizer)get_hf_quantizer)get_module_from_name)auto_conversion)ADAPTER_SAFE_WEIGHTS_NAME DUMMY_INPUTSSAFE_WEIGHTS_INDEX_NAMESAFE_WEIGHTS_NAMEWEIGHTS_INDEX_NAME WEIGHTS_NAMEContextManagers KernelConfigPushToHubMixin cached_filecheck_torch_load_is_safe copy_funchas_fileis_accelerate_availableis_bitsandbytes_availableis_env_variable_trueis_kernels_availableis_torch_flex_attn_availableis_torch_npu_availableis_torch_xpu_availablelogging)GeneralInterfaceis_flash_attention_requestedsplit_attention_implementation)DownloadKwargscreate_and_tag_model_cardget_checkpoint_shard_files)is_flash_attn_greater_or_equal#is_huggingface_hub_greater_or_equalis_sagemaker_mp_enabledis_torch_cuda_available is_tracing)LoadStateDictInfolog_state_dict_report)_CAN_RECORD_REGISTRYOutputRecorder)QuantizationMethod)add_hook_to_module)extract_model_from_parallel)DeviceMeshLike) __version__z1.10F XLA_USE_BF160XLA_DOWNCAST_BF16SpecificPreTrainedModelTypePreTrainedModel)boundT)frozencp\rSrSr%SrSr\S-\S'\"\ S9r \ S-\S'Sr \ S-\S'Sr \ \S 'Sr\S-\S 'Sr\S-\S 'Sr\S-\S 'Sr\ \S 'Sr\R&S-\S'\"\S9r\\S'Sr\S-\S'SrS\S'Sr\ \S'Sr\\\-S-\S'Sr\ S-\S'\S\ 4Sj5r Sr!g)LoadStateDictConfigzY Config for loading weights. This allows bundling arguments that are just passed around. Npretrained_model_name_or_path)default_factorydownload_kwargsuse_safetensorsFignore_mismatched_sizessharded_metadata device_mapdisk_offload_folderoffload_buffersdtype dtype_plan hf_quantizerDeviceMeshLike | None device_meshT weights_onlyweight_mapping disable_mmapreturncURSL$N)rselfs l/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/modeling_utils.py is_quantized LoadStateDictConfig.is_quantizeds  ,,)"__name__ __module__ __qualname____firstlineno____doc__rstr__annotations__r rorrboolrrdictrrrrtorchrrrSrrrlistr#r$rpropertyr__static_attributes__rrrrrs 15!3:4-2>-RO^d*R#'OTD['$)T)$(dTk("Jt "&*t*!OT! $E5;; $T2J2'+L+$++/K(/L$DHND>9:TAH $L$+$ -d--rrrc[=(a? [[RS5=(a [RR 5$)Nis_initialized)_torch_distributed_availablehasattrr distributedrrrr!_is_torch_distributed_initializedrs7$ / E%%'7 8 /    , , .rc[5(a[[RS5(dg[RR 5$)Nget_world_sizer)rrrrrrrr!_get_torch_distributed_world_sizers6 , . .ge>O>OQa6b6b    + + --rc|[5=(a, [[RR SS55S:H$)N LOCAL_RANKz-1r)rintosenvirongetrrris_local_dist_rank_0rs+ , . _3rzz~~lTX7Y3Z^_3__rc#,# SqSv Sqg!Sqf=f7fNTF) _is_quantizedrrrset_quantized_statersM    c#,# SqSv Sqg!Sqf=f7fr)_is_ds_init_calledrrrset_zero3_staters!# "Urrmodel_class_namec#*# UR(dUb USUS3nOSUS3n[U5e[R"5n[R"U5 Sv [R"U5 g![R"U5 f=f7f)z Locally change the torch default dtype to `dtype`, and restore the old one upon exiting the context. If `model_class_name` is provided, it's used to provide a more helpful error message if `dtype` is not valid. Nz% cannot be instantiated under `dtype=z$` as it's not a floating-point dtypez Cannot set `z7` as torch's default as it's not a floating-point dtype)is_floating_point ValueErrorrget_default_dtypeset_default_dtype)rr error_messageoriginal_dtypes rlocal_torch_dtypers  " "  '#$$I%Ptu +5'1hiM'',,.N0 &  //sABA8!B8BBc[R"/5Rn[R"5nX:XaU[R"S5:waU$gU$)z Test if a device context manager is currently in use, or if it is not the case, check if the default device is not "cpu". This is used to infer the correct device to load the model on, in case `device_map` is not provided. cpuN)rtensordeviceget_default_device)device_in_contextdefault_devices r*get_torch_context_manager_or_global_devicersM  R(//--/N* U\\%0 0! ! rcxUR5H\nUR5(dMS[UR5;dM5S[UR5;dMPURs $ [ U5S:Xa[ R $[[UR555R$)zl Returns the first found floating dtype in `state_dict` if there is one, otherwise returns the first dtype. float8_float4_r) valuesrrrlenrfloat32nextiter) state_dictts rget_state_dict_dtypers      Yc!''l%ByX[\]\c\cXdGd77N!  :!}} Z&&() * 0 00r)BOOLU8I8I16U16F16BF16I32U32F32F64I64U64F8_E4M3F8_E5M2pathzstr | os.PathLikec[RRS5(dg[RR [R "U55n[SSS9n[SSU55SS S 9nS S S 5 WH9upEX:Xd*URURS 5S -5(dM4US :Hs $ g!,(df  NN=f![[4a gf=f)aTrue if `path` lives on an hf-mount FUSE filesystem (device string 'hf-mount'). hf-mount's mmap + readahead interaction deadlocks under parallel page-faults, so callers should load the file into memory instead. Linux-only; returns False on other platforms. linuxFz /proc/mountsutf-8encodingc3X# UH n[U5S:dMUSUS4v M" g7f)rrNr).0ps r "_is_on_hf_mount..As*N'>!#a&A+!A$!'>s**c3@# UHoR5v M g7fr)split)rls rrrAs'>2a 2sc[US5$)Nrr)es r!_is_on_hf_mount..Bs c!A$irT)keyreverseN/zhf-mount) sysplatform startswithrrrealpathfspathopensortedrstripOSErrorr)rrealfhentriesdevmps r_is_on_hf_mountr4s << " "7 + +  ww $0 .7 3rN'>2'>N'G4 GCzT__RYYs^c-ABBj(( 4 3 Z    s5=C$C?:C=CC CCC+*C+checkpoint_file map_locationrrc[R"U5nUc [U5nURS5(Ga8U(aoUS:wai[ US5n[ UR 55nSSS5 US:wa3WR5VVs0sHupxXxRU5_M nnnW$[USS9n 0nU R5HnUS:XakU RU5n U R5n U [;a [U n O[SU 35e[R "U R#5U SS 9Xg'MtU R%U5RU5Xg'M UsSSS5 $U(a ['5 0n US:wa[)U5(aS S 0n [R*"U4XS .U D6$!,(df  GNR=fs snnf!,(df  Nl=f) aC Reads a `safetensor` or a `.bin` checkpoint file. We load the checkpoint on "cpu" by default. When `disable_mmap` is True, safetensors files are read fully into memory instead of being memory-mapped. When `disable_mmap` is None (default), it is auto-detected to True on hf-mount FUSE filesystems (see `_is_on_hf_mount`). N .safetensorsmetarbrpt) frameworkz)Cannot load safetensors of unknown dtype )sizerrmmapTrr)rrrendswithr_safe_load_bytesreaditemstorkeys get_slice get_dtypestr_to_torch_dtyperrempty get_shape get_tensorrarr)rrrrcheckpoint_path_fhrkvf_slicek_dtyper extra_argss rload_state_dictr:Msii0O&7 // LF2ot,-chhj9 -u$@J@P@P@RS@Ral!33@R S  $ 71JVVX6)[[^F$..0G"44 27 ;(+TU\T])^__$)KKV5E5E5Gu]c$dJM$%LLO$6$6|$DJM8 7  "Jv*_"="=d^ ::o jL j_i jj9-,T 7 7sF:G # UHn[R"UT5v M g7frresearch)rpatrKs rr6remove_tied_weights_from_state_dict..s!%fFesbiiT&:&:Fes$'z8The weights trying to be saved contained shared tensors z\ which are not properly defined. We found all the potential target tied weights keys to be: zo. This can also just mean that the module's tied weight keys are wrong vs the actual tied weights in the model.)r`rrr) isinstancerridrQrtype get_parameterrRrrarNrranyrSr^clonerd intersection differencerI RuntimeError)rreptrsrptrnames shared_ptrsall_potential_tied_weights_keys error_namesto_delete_namesfoundmatches_pattern shared_namesdisjoint_namesidentical_namesinamesknownunknownrKs @r#remove_tied_weights_from_state_dictrse  " "4 (D"((* f&%,,// F  # #D ) ]]  6 )((.F F  # #D ) "6* + 2 24 8+"15 O *#E Q:3: KO'**?*F&G#KeO'2 '')EEu "%%fFe%f"f"?tz'9QJEs5z)'++D1 &*$2+2D2D2F #S L%d+113 4%4L$M!L!##O4D4 ##O4 w? -9!ww||,IVtu ;DDE^_  -"''.. :|TegnGop33 "ww||1lK\^e>f  !-"''.. :|TkmtGuv33 "ww||1lKbdk>l  " $ :|T`biGjk** "ww||1la  % :|TfhoGpq** "ww||1lK]_f>g  "  *<8I7+S*TU56a9 *<8I7+S*TTYZfgsu|Z}Y~++H*ILWW^^BGGLLR S S8LH.99;DDE^_  -'(97C' g>%"&$4 O#1(&499> +( "" " $%%$,-MNN$'&$O Y )44QS[(r_q(r%)0XN_ahAi5i,75$% ,\7 CDE 'y(ce &<=> ,\7 CDDVgipIqHrrsu$  KK/ ~> ?$0 ! KK/zI^H_` a  77>>) $ $$- ! '"0"$4($&499> + " %00M$o\n$o !-G ) !)-!$! . **" - --7T6_12ei - --}  01N0OP99V8WX::F|U\:]9^^_a   s *E\$A+\$$]!8$]]!rconfigrrcUSLnUGbo[U[5(Ga%US:Xa[US5(a3URb&URn[R SUS35 OU(a SU;aUSnOUUb [ U5nOFUb*USRS5(a[RnO[USSUS 9n[ U5n[R S US 35 O1[[U5(a[[U5nO [S 5e[U[5(a[[U5OUnOI[U[[R45(d[S U35eO[R"5nUbURU5n[U[5(afUR!S[R"55n[U[5(a[[U5OUn[R#SUS35 OUnXlUR$Hn [X)5=n cMXlM X(4$)aFind the correct `dtype` to use based on provided arguments. Also update the `config` based on the inferred dtype. We do the following: 1. If dtype is "auto", we try to read the config, else auto-detect dtype from the loaded state_dict, by checking its first weights entry that is of a floating type - we assume all floating dtype weights are of the same dtype 2. Else, use the dtype provided as a dict or str NautorzWill use dtype=z$ as defined in model's config objectrz.ggufrr%zTSince the `dtype` attribute can't be found in model's config object, will use dtype=z as derived from model's weightsze`dtype` provided as a `str` can only be `'auto'`, or a string representation of a valid `torch.dtype`z`dtype` can be one of: `torch.dtype`, `'auto'`, a string of a valid `torch.dtype` or a `dict` with valid `dtype` for each sub-config in composite configs, but received rzUsing different dtypes per module is deprecated and will be removed in future versions Setting different dtypes per backbone model might cause device errors downstream, therefore setting the dtype=z for all modules.)rmrrrrrrr&rrr:rHrrr update_dtyper warning_once sub_configs) rrrrrrrr main_dtypesub_config_key sub_configs r _get_dtyper1s2"-J  eS ! !67++ 0H"LLEKK/%8\ ]^!g1A&A 0 9#/ 4Z @)5:J1:M:V:VW^:_:_ % %4,Q/fS_& !5Z @KK**/0PR&&u- { .8s-C-CGE5)EED%++#677JJOR 8'')))%0%YYr5#:#:#<= 3=j#3N3NWUJ/T^  !!+ ,= ?  L ,,!&9 9J F) -  rc \rSrSrSr\SSS\R4Sj5r\SSS\R4Sj5rSSS\ S\ 4S jr \ S 5r SSSS \ S \ \S4S\RS -S\ 4 SjjrSSSS\S\S\4SjjrSrg )ModuleUtilsMixiniz@ A few utilities for `torch.nn.Modules`, to be used as a mixin. rrrcB[SUR555$)zu `torch.device`: The device on which the module is (assuming that all the module parameters are on the same device). c38# UHoRv M g7frrrparams rr*ModuleUtilsMixin.device..s@.?ULL.?sr parametersrs rrModuleUtilsMixin.devices @doo.?@@@rcB[SUR555$)zg `torch.dtype`: The dtype of the module (assuming that all the module parameters have the same dtype). c3f# UH'oR5(dMURv M) g7fr)rrrs rr)ModuleUtilsMixin.dtype..s!\->EBYBYB[KEKK->s11rrs rrModuleUtilsMixin.dtypes \T__->\\\rencoder_attention_maskc UR5S:XaUSS2SSS2SS24nUR5S:Xa USS2SSSS24nWRURS9nSU- [R"UR5R -nU$)z Invert an attention mask (e.g., switches 0. and 1.). Args: encoder_attention_mask (`torch.Tensor`): An attention mask. Returns: `torch.Tensor`: The inverted attention mask. Nrr?)dimr*rrfinfomin)rrencoder_extended_attention_masks rinvert_attention_mask&ModuleUtilsMixin.invert_attention_masks " % % '1 ,.DQaQR].S + ! % % '1 ,.DQdTUEU.V ++J*L*LSWS]S]*L*^'+.1P+PTYT_T_`d`j`jTkToTo*o'..rcURnUup4[R"XBS9nUSSSS24RX4S5USSS2S4:*nUR UR 5nUR SUR S:aUUR SUR S- n[R"[R"X4U4X&R S9U/SS9nUSS2SSS2SS24USS2SSSS24-nU$)Nrrrrr<axis) rrarangerepeatr*rshapecatones) input_shapeattention_maskr batch_size seq_lengthseq_ids causal_maskprefix_seq_lenextended_attention_masks r*create_extended_attention_mask_for_decoder;ModuleUtilsMixin.create_extended_attention_mask_for_decoders &&!, ,,z9dD!m,33JANRYZ^`acgZgRhh !nn^%9%9:   Q ."6"6q"9 9+11!4{7H7H7KKN))JJ GPV^o^op K#.aq!m"<~aQUW[]^N^?_"_&&rNrr.rcUc URnUR5S:XaUSS2SSS2SS24nOpUR5S:Xa@[URSS5(a[R X!5nO*USS2SSSS24nO[ SUSURS35eURUS9nS U- [R"U5R-nU$) a Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (`torch.Tensor`): Mask with ones indicating tokens to attend to, zeros for tokens to ignore. input_shape (`tuple[int]`): The shape of the input to the model. Returns: `torch.Tensor` The extended attention mask, with a the same dtype as `attention_mask.dtype`. Nrr is_decoderz!Wrong shape for input_ids (shape z) or attention_mask (shape )rr) rrrHrrrrrr*rrr)rrrrrs rget_extended_attention_mask,ModuleUtilsMixin.get_extended_attention_masks$ =JJE    1 $&4Qa]&C #    !Q &t{{L$77*:*e*e+'+9D$9I*J'3K=@[\j\p\p[qqrs #:"<"<5"<"I#&)@#@EKKPUDVDZDZ"Z&&ronly_trainableexclude_embeddingscU(aHUR5VVs/sH+up4[U[R5(dM&US3PM- nnn[ USS5nU(aSSKnSnUR 5Hup9U(aUW;aMU R(d U(aM.U(a[U WRR5(af[U S5(aU R5n O*[U S5(aU RRn OSn XR5S -U -- nMXR5- nM U$s snnf) a Get number of (optionally, trainable or non-embeddings) parameters in the module. Args: only_trainable (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of trainable parameters exclude_embeddings (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of non-embeddings parameters Returns: `int`: The number of parameters. z.weightis_loaded_in_4bitFrNr@ quant_storagerr)rGrmr EmbeddingrH bitsandbytesnamed_parametersr Params4bitrr@ritemsizenumel) rrrrK module_typeembedding_param_namesrbnb total_paramsr num_bytess rnum_parametersModuleUtilsMixin.num_parameterss% :>:L:L:N%:N%6TR\]hjljvjvRw 4& :N "%$D*=uE  & 002KD!d.C&C""..%E366;L;L)M)Mun55$)$6$6$8  88$)$7$7$@$@ $%  KKMA$5 $AAL KKM1L3"5%s %E ErrFF)rrrrrrrrrrr staticmethodrtuplerrrr rrrrrrsA&A5<<AA]%]%++]] /$5/v/Z`/,''2%) /'/'/'38_/'{{T! /'  /'b*.**bf*sv**rrcl\rSrSrSrSrS\R4SjrS\R4Sjr Sr S r S r g ) EmbeddingAccessMixini"z Base utilities to regroup getters and setters for embeddings. Introduces the `input_layer_embed` attribute, which indicates where the input embeddings come from and where they should be set. embed_tokensrc[USS5n[XS5=nbU$[USS5nUb[X15(a [X15$[USS5nUb[XA5(a [XA5$[USS5nUb%XPLa![US5(aUR5$[SURR S 35e) zv Returns the model's input embeddings. Returns: `nn.Module`: A torch module mapping vocabulary to hidden states. _input_embed_layerrN embeddingsre base_modelget_input_embeddingsu.`get_input_embeddings` not auto‑handled for "; please override in the subclass.)rHrrNotImplementedError __class__r)rrKdefault_embeddingrrers rr)EmbeddingAccessMixin.get_input_embeddings,st1>B")T!: :  G$ $T<6  !gj&?&?:, ,gt,  !5!55' 'T<6  !j&<UkAlAl224 4!` (direct attribute) 2. `self.embeddings.<_input_embed_layer>` (nested embeddings for vision/audio models) 3. `self.model.<_input_embed_layer>` (encoder/decoder models) 4. delegate to the *base model* if one exists 5. otherwise raise `NotImplementedError` so subclasses still can (and should) override for exotic layouts. rrrNrerset_input_embeddingsu.`set_input_embeddings` not auto‑handled for r)rHrrrrrr)rrrKrrers rr)EmbeddingAccessMixin.set_input_embeddingsJst1>B 4   D &#D,==j JwWaOhOh Je ,tWd33e @WUEYEY E '#4t< \:S!\ 4S3j5r?SS4\@S5\AS6\AS7\$\$\A\ 4S84S9\$\$\A\ 4S84S:\@S-4 S;jjrBSS4\@S<\ S!\ 4S=jjrCSS<\ S!\ 4S>jjrDS!\ 4S?jrESS<\ S!\ 4S@jjrFSSA\ S-S<\ SB\ S!\ 4SCjjrGSD\ S-S!\ 4SEjrHSSF\ S-S<\ S!\ 4SGjjrISH\ S-S!\ 4SIjrJ\:S!\ 4SJj5rK\:S!\ 4SKj5rLSSA\ \-SB\ 4SLjjrMSD\ \-4SMjrNSNrOSOrPSSP\ S-4SQjjrQSSP\ S-4SRjjrRSSrSSTrT\*R"5SU5rVSSV\ 4SWjjrW\*R"5\XR"5SX55rZSSY\ S!\4SZjjr[SS\\\ S-S]\ 4S^jjr\S_r]SS`\@S-Sa\@S-Sb\ S!\[TU]"S0UD6 [R"U5R SS5nUR R SS5n[ U5R SS5nURnUbX0lgUbX lgUbXPlgUbX@lgg)Nrr)r)super__init_subclass__inspectget_annotationsr__dict__rr))clsrchild_annotationchild_attributefull_annotationfull_attributers rrK!PreTrainedModel.__init_subclass__s !+F+ #2237;;HdK,,**>4@)-11(DA))  &.   )/   '-   (. )rrc>[TU]5 [U[5(d:[ SUR R SUR R S35eXlURUlURURRS[RS9URl URURR5URlUR#5(a[$R&"U5UlUR R nU[*;aZSSR-[*5S3n[.R0"XPR R 5n[3U5S :aUS nOSnX@lUR6[8[;UR 5'g) NzParameter config in `zt(config)` should be an instance of class `PreTrainedConfig`. To create a model from a pretrained model use `model = z(.from_pretrained(PRETRAINED_MODEL_NAME)`T is_init_checkallow_all_kernels(|rr)rJ__init__rmr! TypeErrorrrr name_or_path%_check_and_adjust_attn_implementation_attn_implementationr-ALLOW_ALL_KERNELS_attn_implementation_internal(_check_and_adjust_experts_implementation_experts_implementation _experts_implementation_internal can_generater*from_model_configgeneration_configrJrrifindallr loss_typerBryr)rrinputsrri loss_groupsrs rr[PreTrainedModel.__init__ss &"233'(?(?'@A NN3344\^   "//594^4^ KK , ,);; 5_5  18<7d7d KK / /8  4     %5%G%G%OD "NN++ L (chh|45Q7K ;0G0GHI9~!%aL  "484L4LS01rc 000sUlUlUlURULaURR b$URR R 5O0UlURRb$URRR 5O0UlURRb$URRR 5O0UlURSS9Ul [UR=(d /5Ul [UR=(d /5Ul [UR=(d /5UlUR5GH3up[!USS5=n(aRURR#UR 5R%5VVs0sH upEUSU3U_M snn5 [!USS5=n(aRURR#UR 5R%5VVs0sH upEUSU3U_M snn5 [!USS5=n(aRURR#UR 5R%5VVs0sH upEUSU3U_M snn5 [!USS5=n(aWURR#UR 5R%5VVs0sHupEUSU3USU3_M snn5 [!US S5=n(aURR#U5 [!US S5=n(aURR#U5 [!US S5=n (dGMURR#U 5 GM6 UR'5 UR)5 gs snnfs snnfs snnfs snnf) a A method executed at the end of each Transformer model initialization, to execute code that needs the model's modules properly initialized (such as weight initialization). It is also used to obtain all correct static properties (parallelism plans, tied_weights_keys, _keep_in_fp32_modules, etc) correctly in the case of composite models (that is, the top level model should know about those properties from its children). NF all_submodels_ep_planrFr=r>all_tied_weights_keysr3r4r1)r=rpr>rrbase_model_pp_plancopybase_model_tp_planbase_model_ep_planget_expanded_tied_weights_keysrqrar3r4r1named_childrenrHupdater) init_weights._backward_compatibility_gradient_checkpointing) rrKrCplanr4r5 tied_keys keep_fp32keep_fp32_strictno_splits r post_initPreTrainedModel.post_initEs=79"b3 t}dm ??d "EI[[EcEcEoDKK::??AuwDMEI[[EcEcEoDKK::??AuwDMEI[[EcEcEoDKK::??AuwDM%)%H%HW\%H%]"%()C)C)Ir%J",/0Q0Q0WUW,X)!$T%;%;%Ar!B!//1LDvz488t8 $$499;CTCTCV%WCV41asmQ&6CV%WXvz488t8 $$499;CTCTCV%WCV41asmQ&6CV%WXvz488t8 $$499;CTCTCV%WCV41asmQ&6CV%WX#F,CTJJyJ**11\e\j\j\l\r\r\t2u\tTXTUdV1QC=TF!A3-3O\t2uv#F,CTJJyJ**11)<#*63QSW#XXX11889IJ"6+>EExEE&&--h7%2*  ;;='&X%W%W3vs O /O O ;O$ c[URS5(a1URRR(a UR$UR $)z* The full tp plan for the model's modules distributed_config)rrrenable_expert_parallelrpr=rs rtp_planPreTrainedModel.tp_planvs< 4;; 4 5 5$++:X:X:o:o== }}rcUR$r)r>rs rpp_planPreTrainedModel.pp_plans }}rr{c LUc0Ulg[U[5(d [S5eUR 5H?up#U[ ;dM[SUSUS[ [ R"5535e UR5VVs/sHupEUPM nnnUR5HbnURSS5nSnUH"n [R"Xy5(dM Sn O U(aMH[R"S US 35 Md Xlgs snnf) Nz&Can only set a dictionary as `tp_plan`z#Unsupported tensor parallel style 'z ' for layer 'z'. Supported styles are *z\d+FTzLayer pattern 'z' does not match any parameters in the model. This rule may not be applied during tensor parallelization, or may lead to dimension mismatches)r=rmrrr)rCrr+rreplacerimatchwarningswarn) rr{ layer_patternparallel_stylerKrXmodel_param_names regex_patternpattern_matchedrs rrrs% <DM $%%EF F.2ZZ\ )M%88 9.9IWdVef,,01D1I1I1K,L+MO.:261F1F1HI1HgdT1HI!YY[M)11#v>M#O/ 88M66&*O0#? %m_5dd) !JsD cfUc0Ulg[U[5(d [S5eXlg)Nz&Can only set a dictionary as `pp_plan`)r>rmrr)rr{s rrrs/ <DM $%%EF F rcV[USS5nUc [S5eURXS9$)zv Potentially dequantize the model in case it has been quantized by a quantization method that support dequantization. rNz?You need to first quantize your model in order to dequantize itr)rHr dequantize)rrrs rrPreTrainedModel.dequantizes8 t^T:  ^_ _&&t&99rcUR(aD[URSS5(a'UR5 [ URS5 ggg)Ngradient_checkpointingF)r?rHrgradient_checkpointing_enabledelattrrs rrz>PreTrainedModel._backward_compatibility_gradient_checkpointingsC  / /GDKKIach4i4i  . . 0 DKK!9 :5j /rtagsc[U[5(aU/nURc/UlUH/nX R;dMURRU5 M1 g)a Add custom tags into the model that gets pushed to the Hugging Face Hub. Will not overwrite existing tags in the model. Args: tags (`Union[list[str], str]`): The desired tags to inject in the model Examples: ```python from transformers import AutoModel model = AutoModel.from_pretrained("google-bert/bert-base-cased") model.add_model_tags(["custom", "custom-bert"]) # Push the model to your namespace with the name "my-custom-bert". model.push_to_hub("my-custom-bert") ``` N)rmrr,rQ)rrtags radd_model_tagsPreTrainedModel.add_model_tagssO, dC 6D ?? " DOC//)&&s+rc URSUR5nURSS5=nb([RS5 X1R:waUOUn[ U[ 5(a[ [U5nURHn[ X5=ncMX6lM SU;aURS5Ul SU;aURS5Ul URSS5n[5/nUb$UR[X0R55 U(aUR[!55 [#5=(a [$(+=(a [&(+n U (ag[R)S 5 S SKn UR-[.R0"5U R2R5[75S 9[95/5 [;U5 U"U40UD6n [=U 5 SSS5 U (aS S KJ n U "W 5 U RC5 W $!,(df  N5=f)z All context managers that the model should be initialized under go here. Args: dtype (`torch.dtype`, *optional*): Override the default `dtype` and load the model under this dtype. r torch_dtypeNz1`torch_dtype` is deprecated! Use `dtype` instead!attn_implementationexperts_implementationrXF@Detected DeepSpeed ZeRO-3: activating zero.init() for this modelrconfig_dict_or_pathr)initialize_weights_zero3)"rTrrrrmrrHrrr_rcrrPrQrrr=r.rrr deepspeedrIinitno_init_weightszeroInitr,rr]rQintegrations.deepspeedr tie_weights) rOrrrrrrrX init_contextsneeds_zero3_initrrers r _from_configPreTrainedModel._from_configs 7FLL1!::mT: :K G    S T"ll2E E eS ! !E5)E%00N%f== J#( 1 !F **0**5J*KF ' $v --3ZZ8P-QF *#JJ':EB&)    !25,,!G H   !6!8 957h n[US5(dMS[U5;dM'UR5(dM> g [US5(a"[R UR S35 g)az Returns whether this model can generate sequences with `.generate()` from the `GenerationMixin`. Under the hood, on classes where this function returns True, some generation-specific changes are triggered: for instance, the model instance will have a populated `generation_config` attribute. Returns: `bool`: Whether this model can generate sequences with `.generate()`. GenerationMixinTrerprepare_inputs_for_generationu6 has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions. - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception). - If you are not the owner of the model architecture class, please contact the model code owner to update it.F)r __bases__rrerwarningr)rObases rrePreTrainedModel.can_generate2s CMM 2 2MMD400 D 1d6G6G6I6I " 37 8 8 NN<<. !   rflash_attn_versiongeneral_availability_checkpkg_availability_checksupported_devices.custom_supported_devicescuda_min_major_versionc VUH(upxU"5(dM[RU5 g U"5(dSUS3n U"5(d[U SUS35eUS:Xa![S5(d[U SUS 35e[ U6up[ S U 55(d[U SUS U S 35eUbl[ 5(a\[RR5upX:a6[U SUS US[RR5SU S3 5egggg)a Checks whether the specified Flash Attention version is supported and if not, searches for the specific reason on why it failed - package import and/or device incompatibility issues. Args: flash_attn_version (`int`): The requested version of Flash Attention. general_availability_check (`Callable`): Checks whether our `is_available` function detects the specific FA version. Failing reasons are then checked for one-by-one. pkg_availability_check (`Callable`): Checks whether the package could theoretically be detected in the environment by the init structures. This is not a sure-fire check as device compatibility with FA is just as important. supported_devices (`tuple[tuple[Callable, str]]`): Essentially a list (for mutable kwargs reasons a tuple) of the supported devices in the format of `(device_availability_check, device_name)`, i.e. a pair of the associated device's name and whether it is available in the environment. custom_supported_devices (`tuple[tuple[Callable, str]]`, *optional*, defaults to `()`): Essentially a list (for mutable kwargs reasons a tuple) of the custom supported devices in the format of `(device_availability_check, info_message)`. These custom devices have custom logic outside the torch ecosystem either via kernels or other packages and hence have early checks for availability. cuda_min_major_version (`int`, *optional*): The minimum major cuda version supported for this version of Flash Attention. This is mostly affecting more recent versions which are more specialized to the features of new hardware. NFlashAttentionzG has been toggled on, but it cannot be used due to the following error:z the package for FlashAttentionz doesn't seem to be installed.rz2.3.3z FlashAttentionz# requires at least version `2.3.3`.c3.# UH o"5v M g7frr)rdevice_availability_checks rr;PreTrainedModel._flash_attn_import_error..ssXr;T466XrzT is not available on CPU. Please make sure you are on any of the supported devices: rFz requires compute capability >= z , but found z with compute capability z.x) rr ImportErrorrrziprqrurcudaget_device_capability)rrrrrrrr info_messageprefacedevice_availability_checks device_namesmajorrXs r_flash_attn_import_error(PreTrainedModel._flash_attn_import_errorXsF8P 3 %(** L)8P *++&'9&:;BCG*++!i>?Q>RRpq$q(1OPW1X1X!WI_=O>**++MN`MabWW[WbWbWpWpVqrnn  Y .9:L9MMefg g %%a(LM_(`a  !t{{$788T[[=Z=Z]^=^##@AS@TU~~  !! =   <=Obcu>v'?:-122'+ ++HI[H\]FFQ]RVX ?$$DEWDXYVV/"Os>HcUR(d"[URRS35e[R R b[RR5S:az[ R"[R5[ R"S5:a>[RS5 [RRRS5 g)a Check the availability of SDPA for a given model. Args: is_init_check (`bool`, *optional*): Whether this check is performed early, i.e. at __init__ time, or later when the model and its weights are fully instantiated. This is needed as we also check the devices of the weights, which are only available later after __init__. This allows to raise proper exceptions early before instantiating the full models if we know that the model does not support the requested attention. a does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet. Please request the support for this architecture: https://github.com/huggingface/transformers/issues/28005. If you believe this error is a bug, please open an issue in Transformers GitHub repository and load your model with the argument `attn_implementation="eager"` meanwhile. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="eager")`rz2.4.1zUsing the `SDPA` attention implementation on multi-gpu setup with ROCM may lead to performance issues due to the FA backend. Disabling it to use alternative backends.FT)r9rrrrrhipr device_countparserrrbackendsenable_flash_sdprrWs r_sdpa_can_dispatch"PreTrainedModel._sdpa_can_dispatchs"">>**+,OO  MM   ) '')A- e//07==3II   y  NN   0 0 7rcrUR5(d"[URRS35eg)z9 Check the availability of Grouped MM for a given model. z1 does not support setting experts implementation.T)_can_set_experts_implementationrrrrs r_grouped_mm_can_dispatch(PreTrainedModel._grouped_mm_can_dispatchs6 3355 7 788ijk krcUR(d"[URRS35e[ 5(d [ S5eg)a  Check the availability of Flex Attention for a given model. Args: is_init_check (`bool`, *optional*): Whether this check is performed early, i.e. at __init__ time, or later when the model and its weights are fully instantiated. This is needed as we also check the devices of the weights, which are only available later after __init__. This allows to raise proper exceptions early before instantiating the full models if we know that the model does not support the requested attention. a does not support an attention implementation through torch's flex_attention. Please request the support for this architecture: https://github.com/huggingface/transformers/issues/34809. If you believe this error is a bug, please open an issue in Transformers GitHub repository and load your model with the argument `attn_implementation="eager"` meanwhile. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="eager")`z]PyTorch Flex Attention requirements in Transformers are not met. Please install torch>=2.5.0.T)r;rrrrhrrs r_flex_attn_can_dispatch'PreTrainedModel._flex_attn_can_dispatchsV''>>**+,tt ,--o  rrrXc [U5upEUbX[USS5n[US9(a=Ub:XV;a5U(aSUS3OUSn[R SUSUSUS 35 Un[U5upEUnS n [US9(aA[ R "5H(n US U 3:XdM[ U S "5(aM&S n O UR(aQU (aJ[5(a;[5(d,[Un[5(aUS:XaS n U(aSU3n[U5(a=U(a [XS9 O [XS9 U (a[R SUS35 U$UR%X5n[US9(a [U5 U$![a+n U (a[!US5n UR#XS9 U eSn A ff=f)a@ Check that the `attn_implementation` exists and is supported by the models, and try to get the kernel from hub if it matches hf kernels pattern. Args: attn_implementation (`str` or `None`): The attention implementation to check for existence/validity. is_init_check (`bool`, *optional*): Whether this check is performed early, i.e. at __init__ time, or later when the model and its weights are fully instantiated. This is needed as we also check the devices of the weights, which are only available later after __init__. This allows to raise proper exceptions early before instantiating the full models if we know that the model does not support the requested attention. allow_all_kernels (`bool`, optional): Whether to load kernels from unverified hub repos, if `attn_implementation` is a custom kernel outside of the `kernels-community` hub repository. Returns: `str`: The final attention implementation to use, including potential fallbacks from sdpa to eager, or from None to sdpa (to potentially eager). Nr<"requested_attention_implementationzpaged|rzNThis model is compatible with the following flash attention implementations: `z"`. Automatically falling back to `z` instead of `z`.Fflash_attention_rTflash_attention_2)rXz/You do not have `flash_attn` installed, using `z%` from the `kernels` library instead!r<rrW)rnrHrmrrrKr+r:rgrirLrjr>rNrMrrrget_correct_attn_implementation) rrrWrXis_pagedbase_implementation compatible_flash_implementationsdefault_flash_implementationapplicable_attn_implementationrequested_original_flash_attn fa_versionrs rr^5PreTrainedModel._check_and_adjust_attn_implementation)s2.)GGZ([%  */6t=`bf/g ,,Pcd4@'OGOf=a@ABTtuvTw-##dfFeGG66R5SSabuavvxz'C#(FGZ([%)<&(-% 'K^ _BGGI (-=j\+JJ@LMijll481J  % %-$&&*,,-GH[-\ *%'',?CV,V16-39:X9Y1Z. 3 4 4 5600Nt1''IJhIij>>*.-.2-Q-Q.. * ,Omn+,JK--# 0!$%8%-F\L]#9tC0E0J0J0L,MPSTmTrTrTtPu,u'vvO_!`O_$=bT"DO_!`%+.DR.H%Hr" II&<=  556H5II~$%Q( W% %  - ---/ "!!!'"a , -$ 4G%,"!!  -sB8$B==C" CC"c\[RRUR5nUb[ US5(dgUR n[ USSS9nUR5nSSS5 [R"SW5(aSU;=(a S U;$g !,(df  N<=f) zDetect whether the class supports setting its attention implementation dynamically. It is an ugly check based on opening the file, but avoids maintaining yet another property flag. N__file__Frrrzclass \w+Attention\(nn.Module\)eager_attention_forwardz&ALL_ATTENTION_FUNCTIONS.get_interface(T) r modulesrrrrrr(rirjrO class_module class_filer6codes r_can_set_attn_implementation,PreTrainedModel._can_set_attn_implementations {{s~~6  w|Z'H'H!** *cG 4668D5 997 > >,4i9aei9i i5 4s B B+c[RRUR5nUb[ US5(dgUR n[ USSS9nUR5nSSS5 SU;$!,(df  SW;$=f)zDetect whether the class supports setting its experts implementation dynamically. It is an ugly check based on opening the file, but avoids maintaining yet another property flag. NrFrrrz@use_experts_implementation)r r!rrrrrr(r"s rr/PreTrainedModel._can_set_experts_implementations{ {{s~~6  w|Z'H'H!** *cG 4668D5-445 4-44s A44 Bc |[U[5(dUO%URSURR5nX0RR:wacUR 5(d-[ RURRS35 O!URUSUS9nX0Rl UR5GH`nX@LdM [U[5(dM!URRURR:wdMQ[URS5(aMnUR 5(d-[ RURRS35 OUn[U[5(agURRHMn[!URU5URLdM(URXdRR5n O UR#U5nXTRl SURlGMc URRHn[!URU5=ncM[U[5(dUOURXgR5n[US5(d~XWR:waoUS/[&R)5-;a/[+S US US [-[&R)5535eXWl [ RS US US35 M[US5(dMU?M g)aS Set the requested `attn_implementation` for this model. Args: attn_implementation (`str` or `dict`): The attention implementation to set for this model. It can be either a `str`, in which case it will be dispatched to all submodels if relevant, or a `dict` where keys are the sub_configs name, in which case each submodel will dispatch the corresponding value. allow_all_kernels (`bool`, optional): Whether to load kernels from unverified hub repos, if `attn_implementation` is a custom kernel outside of the `kernels-community` hub repository. rz does not support setting its attention implementation dynamically, because it does not follow the functional approach based on AttentionInterface (see https://huggingface.co/docs/transformers/en/attention_interface)FrV_attn_was_changedTNrrz"` is not supported for zd. The only possible arguments are "eager" (manual attention implementation)or one of the following: z8We set the attention implementation for the sub-config `z` to `z` without finding the associated sub-model. For this reason we could not check if the model supports it. You may encounter undefined behavior.)rmrrrr_r&rrrrr^rar!rrrrHrr+r rrr)rrrXrequested_implementationrLsub_implementation subconfig_key subconfigs rset_attn_implementation'PreTrainedModel.set_attn_implementations1488 $((T[[-M-MN ! ${{'G'G G4466~~../0\\ ,0+U+U,EUf,V,(=U 9I%y/::$$..$++2G2GG 0 02EFF!==??NN$..7789``*B&!"5t<<-1[[-D-DM&t{{MBiFVFVV5H5L5L$13C3C3X3X6" 2!& .E*3)R)RSe)f&EW$$B6:   2E(J"[[44M$T[[-@@ M&&94@@-,00@^@^_# +>??*.L.LL)'=T=_=_=a1aa(>?Q>RRjkxjyz88<=T=_=_=a8b7ce ?Q;NNRS`Raaghzg{|@@y*=>>%795rc[U[5(dUO%URSURR5nX RR:wa!UR U5nX RlUR5HnX0LdM [U[5(dM URRURR:wdMPUn[U[5(agURRHMn[URU5URLdM(URXSRR5n O URU5nXCRlM g)a Set the requested `experts_implementation` for this model. Args: experts_implementation (`str` or `dict`): The experts implementation to set for this model. It can be either a `str`, in which case it will be dispatched to all submodels if relevant, or a `dict` where keys are the sub_configs name, in which case each submodel will dispatch the corresponding value. rN) rmrrrrcrbrdr!rrrrHr)rrr,rLr-r.s rset_experts_implementation*PreTrainedModel.set_experts_implementationjs24d;; #'++B 0S0ST ! ${{'J'J J'+'T'TUm'n $;SKK 8I%y/::$$..$++2G2GG&>"4d;;)-)@)@ "4;; >)BRBRR1G1K1K -/?/?/W/W2." *A&/%Q%QRd%e"DV  A)(rc4Sn/n[5nSnUR5Hn[U[5(a[ US5(dM+UR 5nUb[ US5(dMR[U5nXs;aMdURU5 URURU55 SnM X l U(a USUl U(d-[RURR S35 gg![ a Mf=f) z Enables the gradients for the input embeddings. This is useful for fine-tuning adapter weights while keeping the model weights fixed. c&URS5 gNT)requires_grad_)rCinputoutputs rmake_inputs_require_gradsMPreTrainedModel.enable_input_require_grads..make_inputs_require_gradss  ! !$ 'rFrNregister_forward_hookTra does not expose input embeddings. Gradients cannot flow back to the token embeddings when using adapters or gradient checkpointing. Override `get_input_embeddings` to fully support those features, or set `_input_embed_layer` to the attribute name that holds the embeddings.)rar!rmrrrrrnrSrQr=_require_grads_hooks_require_grads_hookrrrr)rr;hooks seen_modulesfound_embeddingsrCinput_embeddings embedding_ids renable_input_require_grads*PreTrainedModel.enable_input_require_gradss  (u  llnFv77GFLb#>#@  'w7GI`/a/a./L+   \ * LL)??@YZ [# %%(%*! ',QxD $   >>**+,ww  %'  sD  DDc[USS5nU(dgUHnUR5 M /Ul[US5(aU?gg)z$ Removes the `_require_grads_hook`. r>Nr?)rHremover>rr?)rr@hooks rdisable_input_require_grads+PreTrainedModel.disable_input_require_gradssO4d; D KKM%'! 4. / /( 0rmodalitycRUS;a/SQnO!US:Xa/SQnOUcSS/nO[SU35eUH n[X5(dM[X5s $ URULaE[URS5(a*URR US 9nX@R:waU$U$) aA Best-effort lookup of the *encoder* module. If provided with `modality` argument, it looks for a modality-specific encoder in multimodal models (e.g. "image_encoder") By default the function returns model's text encoder if any, and otherwise returns `self`. Possible `modality` values are "image", "video" and "audio". imagevideo vision_towervisual vision_modelvision_encoder image_toweraudio) audio_tower audio_encoderspeech_encoder text_encoderencoderHUnnrecognized modality, has to be "image", "video" or "audio" but found get_encoderrL)rrrHrr^)rrLpossible_module_namesrK base_encoders rr^PreTrainedModel.get_encoders ) )$o !  $V !  %3Y$? !ghpgqrs s)Dt""t*** ??$ &74??M+R+R??666IL.## rc@US;a/SQnO!US:XaSS/nOUcSS/nO[S U35eUH!n[X5(dM[XU5 g URULa<[URS 5(aURR XS 9 gXlgg) zC Symmetric setter. Mirrors the lookup logic used in `get_encoder`. rNrQrWrXrYNr[r\r] set_encoderr_)rrrrrdre)rr\rLr`rKs rrdPreTrainedModel.set_encoders ) )$o !  %2O$D !  %3Y$? !ghpgqrs s)Dt""G,* ??$ &t 66++G+G$ 'rc/SQnUH n[X5(dM[X5s $ URULa5[URS5(aURR5$U$)ad Best-effort lookup of the *decoder* module. Order of attempts (covers ~85 % of current usages): 1. `self.decoder/self.language_model/self.text_model` 2. `self.base_model` (many wrappers store the decoder here) 3. `self.base_model.get_decoder()` (nested wrappers) 4. fallback: raise for the few exotic models that need a bespoke rule )language_model text_modeldecoder text_decoder get_decoder)rrHrrk)rr`rKs rrkPreTrainedModel.get_decoder sc!\)Dt""t*** ??$ &74??M+R+R??..0 0 rc/SQnUH!n[X5(dM[XU5 g URULa>[URS5(aURRU5 gXlgg)zC Symmetric setter. Mirrors the lookup logic used in `get_decoder`. )rgrhriN set_decoder)rrrrnre)rrir`rKs rrnPreTrainedModel.set_decoder" sh !L)Dt""G,* ??$ &t 66++G4$ 'rc [URS5(a URR=(d SnO[URS5(aURRnOW[URS5(aURRnO%[ URR 5SS5n[U[R[R[R[R[R[R45(ak[ USS5b.[R "UR"R%5SUS9 UR&b![R("UR&5 gg[U[R*5(ay[R "UR"SUS9 UR,bK[ UR"S S 5(d.[R("UR"UR,5 ggg[U[R.5(aUR15 g[U[R2[R4[R6[R845(d4S UR:R<;dS UR:R<;a[ USS5b [R>"UR"5 [ US S5b [R("UR&5 [ USS5ba[R("UR@5 [R>"URB5 [R("URD5 ggSUR:R<;a[US5(aURFS:wa[HURFO URJnU"UR5upE[RL"URNU5 [RL"URPU5 ggg)aD Initialize the weights. This is quite general on purpose, in the spirit of what we usually do. For more complex initialization scheme, it should be overridden by the derived `PreTrainedModel` class. In case a model adds an explicit `nn.Parameter`, this method should also be overridden in order to initialize it correctly. initializer_rangeg{Gz?init_stdinitializer_factorweightNgmeanstd_is_hf_initializedF LayerNormRMSNormbias running_meanRotaryEmbeddingoriginal_inv_freqdefault))rrrqrrrsrHget_text_configrmrLinearConv1dConv2dConv3dConvTranspose1dConvTranspose2drnormal_rtfloatr{zeros_r padding_idxMultiheadAttention_reset_parameters GroupNorm BatchNorm1d BatchNorm2d BatchNorm3drrones_r| running_varnum_batches_tracked rope_typerOcompute_default_rope_parameterscopy_inv_freqr~)rrCrwrope_fn buffer_valuerXs r _init_weightsPreTrainedModel._init_weights3 s 4;; 3 4 4++//74C T[[* - -++&&C T[["6 7 7++00C$++5579LdSC fryy"))RYY 2K]K]_a_q_qr s svx.: V]]002#F{{& FKK('  - - LLSc :!!-gfmmMach6i6i FMM&*<*<=>7j-  5 5 6 6  $ $ & v bnnbnnbnn] ^ ^f..777F,,555vx.: 6==)vvt,8 FKK(v~t4@ F//0 6--. F667A &"2"2";"; ;Pc@d@d##y0$F$4$45;;  &fmm4OL JJv 5 JJv// >Ae ;rrc[USS5(agU(aP[SURSS955(a,[SURSS955(aSUlgUR U5 SUlg)z= Initialize the weights if they are not already initialized. rxFNc3<# UHn[USS5v M g7f)rxFNrHrs rr6PreTrainedModel._initialize_weights..y snMmEGE#7??Mms)recursec3F# UHnUcM[USS5v M g7f)NrxFr)rbuffers rrrz s);F= 4e<<;s!!T)rHallrbuffersrxr)rrCrs r_initialize_weights#PreTrainedModel._initialize_weightsm s 6/ 7 7  nVM^M^glM^Mmnnn$nnUn; )-F %  6"$(!rcv^[[RRS5(dbS[RS[[R[ /S4S[ 4U4Sjjm[ [RRST5 [US5nU"URUR55 g)a This is equivalent to calling `self.apply(self._initialize_weights)`, but correctly handles composite models. This function dynamically dispatches the correct `init_weights` function to the modules as we advance in the module graph along the recursion. It can handle an arbitrary number of sub-models. Without it, every composite model would have to recurse a second time on all sub-models explicitly in the outer-most `_init_weights`, which is extremely error prone and inefficient. smart_applyrCrNrc>UR5H6n[U[5(aT"X3RU5 M-T"X1U5 M8 U"X5 U$r)childrenrmrr)rCrrchildrs rr7PreTrainedModel.initialize_weights..smart_apply sJ#__.E!%99#E+D+DnU#E~> / 6* r) rrrr'rrrrHrr)rsmart_apply_fnrs @rinitialize_weights"PreTrainedModel.initialize_weights suxx 66 BII 8RYY# UHnTRU5v M g7fr)r)rr4common_case_regexs rrAPreTrainedModel.get_expanded_tied_weights_keys.. s"_3^a &&q))3^s!^c2>[R"TU5$rrh)x source_names rr@PreTrainedModel.get_expanded_tied_weights_keys.. BIIk14Mrc2>[R"TU5$rrh)r target_names rrr rrrzAThere is an issue with your definition of `tie_weights_keys` for :z . We found z to tie into )rGrmrrvr)rxrErHrricompilerr+rrsr named_buffersrfilterrrrr )rroexpanded_tied_weightsprefixrLsubmodel_tied_weightsr4r5 tied_mappingrrXall_param_names source_params target_paramstarget_nsource_nrrrs @@@rrv.PreTrainedModel.get_expanded_tied_weights_keys sd $& !%)%7%7%7%O!i99,5,T,Tch,T,i)|I^IdIdIf1IfvhasOxq_)>PU)>)VW)V1)VW,,e,D[ D$!AD[  )5(:(:(< $K +K +K"6*M#_`M"6*M#_`M &*=)A-}%M(::a? WXcWddefqers -mM?L '*-}9M&N"99;;6K6U)37?(3'O!)=6%$o12X[ s I7IIT missing_keysrecompute_mappingcU(d URnOURSS9n[UR55n[ U5GH{unupVUbSnXa;nXQ;n U(a{U (at[ R "URU5URU55(d9[RSUSUS35 URRU5 MOcU(d U (aXepeORU(dKU (dDX4S-SHupX:XdM X;n U (dMU n O Sn[RS USUS 35 URU5n S U;a&URS S5upURU5nOUnUn[UX5 URUU 5 UcGM`W(dGMjUR!U5 GM~ g) a Tie the model weights. If `recompute_mapping=False` (default when called internally), it will rely on the `model.all_tied_weights_keys` attribute, containing the `{target: source}` mapping for the tied params. If `recompute_mapping=True`, it will re-check all internal submodels and their config to determine the params that need to be tied. This is the default when `model.tie_weights()` is called on its own, outside of `__init__`, and `from_pretrained`, in case the config values were changed somewhere. Note that during `from_pretrained`, tying is *symmetric*: if the mapping says "tie target -> source" but `source` is missing in the checkpoint while `target` exists, we *swap* source and target so we can still tie everything to the parameter that actually exists. TrnNzDThe tied weights mapping and config for this model specifies to tie z to z, but both are present in the checkpoints with different values, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning.rFzYThis checkpoint seem corrupted. The tied weights mapping for this model specifies to tie zk, but both are absent from the checkpoint, and we could not find another related tied weight for those keysrF)rqrvrr) enumeraterequalrprrrTget_parameter_or_bufferr get_submoduler _adjust_biasdiscard)rrrr|itarget_param_namesource_param_nameremove_from_missingsource_is_theretarget_is_there target_backup source_backuptarget_backup_is_there source_param parent_namerKrs rrPreTrainedModel.tie_weights s!22I;;$;OI*+ 9B99M 5A5!'&*#"3"G"3"G#!;;t'9'9:K'LdN`N`arNsttbctbuuy012 22667HI u)_;L'8(8Aa%'8J4 )=5B5V2 654A 1 %9K /4+w016G5HI__ 778IJL''$5$<$= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc mean_resizing (`bool`): Whether to initialize the added embeddings from a multivariate normal distribution that has old embeddings' mean and covariance or to initialize them with a normal distribution that has a mean of zero and std equals `config.initializer_range`. Setting `mean_resizing` to `True` is useful when increasing the size of the embeddings of causal language models, where the generated tokens' probabilities won't be affected by the added embeddings because initializing the new embeddings with the old embeddings' mean will reduce the kl-divergence between the next token probability before and after adding the new embeddings. Refer to this article for more information: https://nlp.stanford.edu/~johnhew/vocab-expansion.html Return: `torch.nn.Embedding`: Pointer to the input tokens Embeddings Module of the model. Nrr modifier_rank) _resize_token_embeddingsrrr.rrGatheredParametersrtrrr vocab_sizer)rrrr model_embedsrrrs rresize_token_embeddings'PreTrainedModel.resize_token_embeddingsu sH44^Yfg  !&8&@ t^4V9J9JRV9V % ' ' 22<3F3FVZ2[)0066q9 \[&,,2215J4> ##%0$ \[s :C-- C;cUR5nURXAX#5n[US5(aURn[ XV5 UR R nURU5 URU5 [US5=(a URSLnUbz[5(aRU(dKSSK n U RRUR SS9 UR RSnSSS5 OUR RSnUR5bUR5n [!U ["R$R&5(aURXUS9n OUR)XUS9n [U S5(aU Rn[ X5 U R R n U RU 5 UR+U 5 UR5$!,(df  N=f)N_hf_hookrrr)r)r_get_resized_embeddingsrrr|rtrr8rrr.rrrrr!rmrrr_get_resized_lm_headr%) rrrrold_embeddingsr$rIold_embeddings_requires_gradrr old_lm_head new_lm_headold_lm_head_requires_grads rr(PreTrainedModel._resize_token_embeddings s22455 ,>  >: . .!**D ~ 4'5'<'<'J'J$%%&BC !!.1t^4V9J9JRV9V   ))++L ^^66~7L7L\`6a%3%:%:%@%@%CNba"0!6!6!= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc mean_resizing (`bool`): Whether to initialize the added embeddings from a multivariate normal distribution that has old embeddings' mean and covariance or to initialize them with a normal distribution that has a mean of zero and std equals `config.initializer_range`. Setting `mean_resizing` to `True` is useful when increasing the size of the embeddings of causal language models, where the generated tokens' probabilities will not be affected by the added embeddings because initializing the new embeddings with the old embeddings' mean will reduce the kl-divergence between the next token probability before and after adding the new embeddings. Refer to this article for more information: https://nlp.stanford.edu/~johnhew/vocab-expansion.html Return: `torch.nn.Embedding`: Pointer to the resized Embedding Module or the old Embedding Module if `new_num_tokens` is `None` Nz5Asking to pad the embedding matrix to a multiple of `z@`, which is not and integer. Please make sure to pass an integerrrzYou are resizing the embedding layer without providing a `pad_to_multiple_of` parameter. This means that the new embedding dimension will be a.. This might induce some performance reduction as *Tensor Cores* will not be available. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tcrrzOld embeddings are of type , which is not an instance of zj. You should either use a different resize function or make sure that `old_embeddings` are an instance of rFrzThe new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`)rmrrrtrrrrrr.rrrr#rrrr\rorrrr(_init_added_embeddings_weights_with_meanrrr) rrrrrrrold_num_tokensold_embedding_dimr$added_num_tokensnparamss rr'PreTrainedModel._get_resized_embeddings sfV  )0#66 KL^K_``a%!/!6!6!3H3HX\2]4B4I4I4N4N4P1^]1?0E0E0J0J0L -N ^ +4N4P4P,: )! !.",,77-d>.B-CCabdbnbnaopLL>$   !((// ''--    *=   ~ .  ,   =  .> )++L ^^668M8M7N^b6cAA&dc =="N  / % ' ' $++^-B-BCF2262K4B4I4I4N4NrPQrSTu4U%%**2A2q51LK1?0E0E0J0J2A2q50QN ! ! & &rr1u - & ' ' $++^-B-BCF2262K(6(=(=%0>0E0E0J0J0P0PQR0S-"--9~PQ?QUcUoUo>o15N.L *8)>)>)C)CN ! ! &,:,A,A,F,F,L,LQ,ON )))5>A;MQ_QkQk:k-1*w^]^dc$LKLKs1Q<R =R A!R2< R  R R/2 Sr transposedc hUcU$[US5=(a URSLn[5(aU(d}SSKnURR UR SS9 U(dUR R5O'UR R5R5upxSSS5 OKU(dUR R5O'UR R5R5upxWU:Xa[5(dX!l U$[U[R5(d:[S[U5S[RS[RS35eU(dWU4OUW4n URSLn [R"U U UR R UR R"S .6n X':aU(dUR%U 5 OX':aU(a[&R)S 5 X'- n [5(a~U(dwSSKnUR /n U (aXR/- n URR U SS9 UR+XXX5 U (aUR-XU 5 SSS5 O,UR+XXX5 U (aUR-XU 5 [/Xr5n[5(apU(diSSKnUR URU R U R/n URR U SS9 UR1XXU 5 SSS5 OUR1XXU 5 [3U S S 5 U $!,(df  GN=f!,(df  N=f!,(df  N@=f) a Build a resized Linear Module from a provided old Linear Module. Increasing the size will add newly initialized vectors at the end. Reducing the size will remove vectors from the end Args: old_lm_head (`torch.nn.Linear`): Old lm head liner layer to be resized. new_num_tokens (`int`, *optional*): New number of tokens in the linear matrix. Increasing the size will add newly initialized vectors at the end. Reducing the size will remove vectors from the end. If not provided or `None`, just returns a pointer to the input tokens `torch.nn.Linear` module of the model without doing anything. transposed (`bool`, *optional*, defaults to `False`): Whether `old_lm_head` is transposed or not. If True `old_lm_head.size()` is `lm_head_dim, vocab_size` else `vocab_size, lm_head_dim`. mean_resizing (`bool`): Whether to initialize the added embeddings from a multivariate normal distribution that has old embeddings' mean and covariance or to initialize them with a normal distribution that has a mean of zero and std equals `config.initializer_range`. Setting `mean_resizing` to `True` is useful when increasing the size of the embeddings of causal language models, where the generated tokens' probabilities will not be affected by the added embeddings because initializing the new embeddings with the old embeddings' mean will reduce the kl-divergence between the next token probability before and after adding the new embeddings. Refer to this article for more information: https://nlp.stanford.edu/~johnhew/vocab-expansion.html Return: `torch.nn.Linear`: Pointer to the resized Linear Module or the old Linear Module if `new_num_tokens` is `None` Nrrrz#Old language model head is of type rzg. You should either use a different resize function or make sure that `old_lm_head` are an instance of rF)r{rraThe new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`rxT)rrr.rrrrtr#rrrmrrr\ror{rrrrr%_init_added_lm_head_weights_with_mean"_init_added_lm_head_bias_with_meanr!_copy_lm_head_original_to_resizedr)rrrr rrrrold_lm_head_dimnew_lm_head_shapehas_new_lm_head_biasrrrnum_tokens_to_copys rr$PreTrainedModel._get_resized_lm_headv sLH  ! t^4V9J9JRV9V % ' ' 22;3E3EUY2Z5?K&&++-[EWEWEYEYE[E`E`Eb0[Z 2< ""'')ASASAUAUAWA\A\A^ ,N ^ +4N4P4P'5 $ +ryy115d;6G5HHfgigpgpfqrII;a! FP_n=VdfuUv*//t; ii %%%,,$$**    *=   { +  ,   =  .> )++L %,,-'//00F^^66vT6R>>#/Sc,?? Zjk SR::oO_(;;KVfg @ % ' ' !((+*:*:K)covariance_matrix) sample_shaper<r)rtrr*rrrvTrpositive_definitecheckr distributionsmultivariate_normalMultivariateNormalsamplerr) rrr$rrold_embeddings_weightmean_embeddingsold_centered_embeddings covarianceepsilonis_covariance_psd distributions rr8PreTrainedModel._init_added_embeddings_weights_with_mean s]!/ 5 5 : : = =emm L**%:C"7"I,..1HH>Y '99??@TUYY[  ..BBUUZ3GVLFREXEX.0FYFb&&,,-  ! ! & &r,<'<'>'A B  a(//0@!DGGH]H]HcHcd  ! ! & &r,<'<'>'A BrcU(a^URRRURlURRRURlURXXE5 U(a_URRRURlURRRURlggr)rtrrr)rrrrrrr s rr 5PreTrainedModel._init_added_lm_head_weights_with_mean s &1&8&8&=&=&?&?K   #&1&8&8&=&=&?&?K   # 55kP^q &1&8&8&=&=&?&?K   #&1&8&8&=&=&?&?K   # rcZ[R"URRS[RS9n[R "URRSS9R [R5nURRSU-SRUSU-S9 g)Nr)rrrr<rru)rrvr{rrrwr*r)rrrr bias_meanbias_stds rr 2PreTrainedModel._init_added_lm_head_bias_with_mean( sJJ{//441EMMR 99[--22;>>u}}Mb#3356>>ISWZbSb>crctU(d=URRSU2SS24URRSU2SS24'OI>P>P>U>UViWiViklVl>mK   # #$7%7$7$: ;>I>P>P>U>UVWYlZlYlVl>mK   # #A':(:':$: ; 9D9I9I9N9NObPb9cK   ! !"5#5 6 rnew_num_position_embeddingsc |[SURSURSURRS35e)Nz4`resize_position_embeddings` is not implemented for B`. To implement it, you should overwrite this method in the class  in `modeling_.py`rrr)rr/s rresize_position_embeddings*PreTrainedModel.resize_position_embeddings: sH!B4>>BRS226..1APTP^P^PiPiOjjn p  rc |[SURSURSURRS35e)Nz1`get_position_embeddings` is not implemented for r1r2r3r4rs rget_position_embeddings'PreTrainedModel.get_position_embeddings@ sH!??OP226..1APTP^P^PiPiOjjn p  rc[5[R"S5:waUR5 UR SS9 g)z Initialize and tie the weights if needed. If using a custom `PreTrainedModel`, you need to implement any initialization logic in `_init_weights`. rF)rN)rrrrrrs rryPreTrainedModel.init_weightsF s5 6 75<<;O O  # # % 51rc"UR(d"[URRS35eUcSS0n[R "[ 40UD6nS[R"UR5R;nU(dURSUS9 O8UR[ URSS95 [RS 5 URS :HnU=(d [US S5nU(aUR!5 gg) a Activates gradient checkpointing for the current model. We pass the `__call__` method of the modules instead of `forward` because `__call__` attaches all the hooks of the module. https://discuss.pytorch.org/t/any-different-between-model-input-and-model-forward-input/3690/2 Args: gradient_checkpointing_kwargs (dict, *optional*): Additional keyword arguments passed along to the `torch.utils.checkpoint.checkpoint` function. z) does not support gradient checkpointing.N use_reentrantFrT)enablegradient_checkpointing_funcrVYou are using an old version of the checkpointing format that is deprecated (We will also silently ignore `gradient_checkpointing_kwargs` in case you passed it).Please update to the new format on your modeling file. To use the new format, you need to completely remove the definition of the method `_set_gradient_checkpointing` in your model.r-_hf_peft_config_loaded)r?rrr functoolsr rrL signature_set_gradient_checkpointingrapplyrrr.rHrE)rgradient_checkpointing_kwargsr?_is_using_old_formatneeds_embedding_gradsenable_input_gradss rr-PreTrainedModel.gradient_checkpointing_enableR s33 7 788abc c ( 0-r?c$Sn[US5(aX lXlSnUR5H2n[US5(dM[ USU5 [ USU5 SnM4 U(d"[ UR RS35eg)NFrT_gradient_checkpointing_funcz is not compatible with gradient checkpointing. Make sure all the architecture support it by setting a boolean attribute `gradient_checkpointing` to modules of the model that uses checkpointing.)rrMrr!rrrr)rr>r?is_gradient_checkpointing_setrCs rrE+PreTrainedModel._set_gradient_checkpointing| s(-% 41 2 20K -*0 ',0 )llnFv788 >@[\ 8&A04- % ->>**+,]] -rcdUR(a|S[R"UR5R;nU(dURSS9 O8[ R S5 UR[URSS95 [USS5(aUR5 gg)z; Deactivates gradient checkpointing for the current model. rF)r>rAr@rBN) r?rLrDrErrrrFr rHrJ)rrHs rgradient_checkpointing_disable.PreTrainedModel.gradient_checkpointing_disable s  / /$+g.?.?@`@`.a.l.l#l '000>L 74#C#C5QR 415 9 9  , , . :rcB[SUR555$)zD Whether gradient checkpointing is activated for this model or not. c3`# UH$n[US5=(a URv M& g7f)rN)rr)rms rr. s)m^lYZ7167TA If a single weight of the model is bigger than `max_shard_size`, it will be in its own checkpoint shard which will be bigger than `max_shard_size`. variant (`str`, *optional*): If specified, weights are saved in the format model..safetensors. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). save_peft_format (`bool`, *optional*, defaults to `True`): For backward compatibility with PEFT library, in case adapter weights are attached to the model, all keys of the state dict of adapters needs to be prepended with `base_model.model`. Advanced users can disable this behaviours by setting `save_peft_format` to `False`. save_original_format (`bool`, *optional*, defaults to `True`): For backward compatibility with the previous versions of `transformers` you can save the checkpoint with its reverse mapping. The reverse mapping needs to exists even if the model was loaded from a None legacy checkpoint. kwargs (`dict[str, Any]`, *optional*): Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. NrrBFrzThe model is quantized with z and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.z0.31.4z[Saving a model with tensor parallelism requires `huggingface_hub` version 0.31.4 or higher.zProvided path (z#) should be a directory, not a fileT)exist_okcommit_messagerepo_idr< create_prr`r!formatrFrFSDP)rzhDetected adapters on the model, saving the model in the PEFT format, only adapter weights will be saved.)rzTo match the expected format of the PEFT library, all keys of the state dict of adapters will be prepended with `base_model.model`.zbase_model.model.zMultiple active adapters detected, saving multiple active adapters is not supported yet. You can save adapters separately one by one by iteratively calling `model.set_adapter(adapter_name)` then `model.save_pretrained(...)`r hf_device_maprdiskz}Attempting to save a model with offloaded modules. Ensure that unallocated cpu memory exceeds the `shard_size` (50GB default)z.binz {suffix}.binrz{suffix}.safetensors)filename_patternr\total_parameters)metadata weight_maprz(.*?)-\d{5}-of-\d{5}zWriting model shards)descr)rjzModel weights saved in wrrr)indent sort_keys z:The model is bigger than the maximum size per checkpoint (z) and is going to be split in z^ checkpoint shards. You can find where each parameters has been saved in the index located at )rz README.md)rarrc)^rHrmrSis_serializablerquantization_config quant_method_tp_sizersrrrrrerrormakedirsrrTrseprrb_get_files_timestampsget_state_dict_and_metadata unwrap_modelrrrrr removeprefix architectures _auto_classr(save_pretrainedrergrget_adapter_state_dictr)active_adaptersr peft_configrrrarfrrrIS_SAGEMAKER_MP_POST_1_10smpstatemodule_managertranslate_functionsr8rFr= _device_meshrr&rZrrWrrrr rjtensor_to_filenamelistdirrrirrfilename_to_tensors fullmatchrHrktqdmrror6 contiguoussafe_save_filerYrjsondumpswriterpr,save_upload_modified_files)2rrYrZrr[r\rrr]r^rrBrquantization_serializablesave_directory_pathrarbrcfiles_timestampsrj model_to_saverpeft_state_dictr ractive_adaptercurrent_peft_config is_offloaded smp_to_hfrX ignore_keyrrhstate_dict_splitindexr full_filenameweights_no_suffixfilename_no_suffixreg shard_file tensor_namesshard_state_dict tensor_namerpath_to_weightssave_index_filer6content model_cards2 rr~PreTrainedModel.save_pretrained sv  #7O!(/G!Ot^T:  $ qL+)N qS_SoSoSq "  #,BKd.|/O/O/\/\.]^uu  == $-PQY-Z-Zm  77>>. ) ) LL?>*::]^ _  NT2 ii7 #ZZ(8$?Njj,?,E,Ebggkk,RSU,VWG ;6I!'CDCFCKKG#99.I   ##/#K#KD#Q J!%T* ##%(Z%5%5c%:1%= "/<.E.E.N.N.[.[\b.c-d *    ' tDKK H )$$44^D  ""//??O% ~+AAZAX #KK^')O&0&6&6&8 UEJ*;C5(AB'9!0J!%!5!5!7~&*$u"0!2&*&6&6~&F##33NC  &113J D/ * *C**113459$,,33554CUCUC\C\C^9^L MM:  % $ # 8 8 L L 1&z2 !M  ' ' 3":: +":.; == $3J tO`O`bfbobopJ9]S  (>1-LJ&,L' g>L4L'//GOOP^`vw= )9.   & &/1D1D1FdJZJcJcd.AAE  >2HGGLLBM!- 4 4VR @ H HY[ \ "*!1!1&"!=!E!EnVX!Y **45C##$566GGNN=11$4$H$HH#OMM"45A -(#3()0  0 0 6 6 8?U) $J ww||NJ?H! + # 4  FMM$6$6&$@5m[QF170A0A0C - , +X I /) 2 = ggll><HO KK1/1BC D5O ggll><Y`;abOosW=**U1EL > KKL^L\] 0 D DEFG$$3#4A7  27DOOSXYJ OOBGGLLE F  ' ' -# (  >=s +b  bc>URb URO/nURS/5n[U[5(aU/nUHnXS;dM UR U5 M U(aX2S'[ TU]"U0UD6$)Nr)r,rrmrrQrJr[)rrrr tags_kwargsrrs rr[PreTrainedModel.push_to_hub sw"&//"=t2jj, k3 ' '&-KC C  !6Nw"D3F33rc[SUR555nU(a$[SUR555nX#-nU$)a Get the memory footprint of a model. This will return the memory footprint of the current model in bytes. Useful to benchmark the memory footprint of the current model and design some tests. Solution inspired from the PyTorch discussions: https://discuss.pytorch.org/t/gpu-memory-that-model-uses/56822/2 Arguments: return_buffers (`bool`, *optional*, defaults to `True`): Whether to return the size of the buffer tensors in the computation of the memory footprint. Buffers are tensors that do not require gradients and not registered as parameters. E.g. mean and std in batch norm layers. Please see: https://discuss.pytorch.org/t/what-pytorch-means-by-buffers/120266/2 c3b# UH%oR5UR5-v M' g7frr=r@rs rr7PreTrainedModel.get_memory_footprint.. s%YGXe.."U%7%7%99GX-/c3b# UH%oR5UR5-v M' g7frr)rbufs rrr s"Y.3<<>C,<,<,>>.r)sumrr)rreturn_buffersmemmem_bufss rget_memory_footprint$PreTrainedModel.get_memory_footprint s@YtGXYY Y$,,.YYH.C rc>[USS5[R:XaySSKJn [ TU]"U0UD6 UR5HMn[XC5(dM[U5S:aUSnOURSS5nUR U5 MO U$[USS5[R:Xa[USS5(a [S5e[ TU]"U0UD6$) Nquantization_methodr HQQLinearrris_loaded_in_8bitFzCalling `cuda()` is not supported for `8-bit` quantized models. Please use the model as it is, since the model has already been set to the correct devices.) rHr{HQQhqq.core.quantizerrJrr!rmrrBITS_AND_BYTESr)rrrrrCrrs rrPreTrainedModel.cuda s 4. 59K9O9O O 3 GL$ )& ),,.f004y1}!%a!'Hf!=KK' )K 4. 59K9Z9Z Zt0%88 sw|T,V,,rc\>SU;nU(d,UH&n[U[R5(dM$Sn O [USS5[R :XaSSKJn [T U]$"U0UD6 UR5HXn[Xe5(dMSU;aUSnOUSnSU;aUSnO U(aWnOSnUbXl URU5 MZ U$U(a*[USS5[R:Xa [S5e[USS5[R:Xa@U(a [S5e[US S 5(a[!S 5(d [S 5eO1[USS5[R":XaU(a [S 5e[T U]$"U0UD6$)NrTrrrrzBCasting a Quark quantized model to a new `dtype` is not supported.zYou cannot cast a bitsandbytes model in a new `dtype`. Make sure to load the model using `from_pretrained` using the desired `dtype` by passing the correct `dtype` argument.rFz0.48zsYou need to install `pip install bitsandbytes>=0.48.0` if you want to move a 8-bit model across devices using to().zYou cannot cast a GPTQ model in a new `dtype`. Make sure to load the model using `from_pretrained` using the desired `dtype` by passing the correct `dtype` argument.)rmrrrHr{rrrrJr*r! compute_dtyperQUARKrrreGPTQ) rrrdtype_present_in_argsargrrCrrrs rr*PreTrainedModel.tos!(6 1$c5;;//,0) 4. 59K9O9O O 3 GJ ' ',,.f006)!'!1!%a&( &w. # $(/4,KK'#)$K WT3H$%OSeSkSk%kab b 4. 59K9Z9Z Z$ P t0%88AZ[aAbAb JT0$ 7;M;R;R R$ Hwz4*6**rcX>[USS5(a [S5e[TU] "U6$)NrFz`.half()` is not supported for quantized model. Please use the model as it is, since the model has already been casted to the correct `dtype`.)rHrrJhalfrrrs rrPreTrainedModel.halfDs6 4 / /I  7<& &rcX>[USS5(a [S5e[TU] "U6$)NrFz`.float()` is not supported for quantized model. Please use the model as it is, since the model has already been casted to the correct `dtype`.)rHrrJrrs rrPreTrainedModel.floatNs6 4 / /I  7=$' 'rrrrc[XR5[R"5[ 5/nU(aUR [ 55 [5(aSSKnU(dlU(de[RS5 UR[R"5URR[5S9[!5/5 U$U(a/UR["R$"S5['5/5 U$UR["R$"S5[R("5/5 U$)Nrrrr)rrrno_tie_weightsrPrQr=r.rrrrIrrrr,rrrrmeta_device_safe_creation_ops)rOrrrrXrrs rget_init_context PreTrainedModel.get_init_contextXs +5,,?ATATAVXeXgh   !6!8 9 % ' '  (: ^_$$,,.!++@P@R+S')$$ell6&:=0.9.0. Please install the latest version with `pip install -U kernels`r)use_kernel_mappingr)$register_kernel_mapping_transformersN)inherit_mappingTF) rgrkernelsrintegrations.hub_kernelsrrmr^sanitize_kernel_mappingcreate_compatible_mappinguse_local_kernelkernel_mapping use_kernels)rrrrrrs rset_use_kernelsPreTrainedModel.set_use_kernelss ')) B 3 V 0 2(Z |-T-T55d;77= '4&D&D"D' (D(DVef'+D$gf$( $D  gfs B22 Cr) rrrrrrrrr fusion_configrrOrrrrrrrrrrc  URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5nURS S5nURS S5nURS S5nURS S5nURSS5nURSS5nURSS5nURSS5nURSS5nURS05=(d 0R5nURSS5n URSS5n!URSS5n"URSS5n#URSS5n$URSS5n%URSS5n&URSS5n'URSS5n(URSS5n)URS S5n*URS!S5n+U%bU#cS"n#S#Hn,URU,S5n-M UbUbUOUnUcS"n[5(a U(dS$nUUUUUUUS%.n.0U.ES&U0En/UbUcU"b [S'5eUS":XaC[ [ R RS(S)55(a[RS*5 U#cU$b[U#U$U&US+9unn&n$U"b[5(d [S,5eUc0n[UU/40UD6un0nn[U5nS-S.US/.n1UbUU1S0'[U[5(duUbUOUn2UR n3U3c[UR"S135eU3R$"U24S$U"UUS2.U.DUD6unn4SU4;aU4RS5 U4RSU5nO%[R&"U5nUn4[)USU5nUU/S&'S3U;aURS35UlS4U;aURS45Ul[/UUUU U15un5nnU"(aKU5b [S55eUb:[U[05(aS6UR35;dS6U;a [5S75eU*bU)(d[R7S85 S$n)[9UUU"U U/U1UR;5[)US9S5US:9 un6n7U5SLn8[=UU6UU7XU55unnU"(a<S;SS$W:US?9S@nXl#U b[R&"U 5Ul$[)USAS5n U bS;SBK%J&n; U;"XU 5 UROUU8[PU(5n<[R&"U5n[SU<5 U"U/U Q70U4D6n=[UU=5 U5bU5RWU=UUU6U)SC9 SSS5 W=RYU5n>[[U=U+U55n?[\(aU&b[_U=U#U%U&U$5n=Ub[aU=UUU55n[cUUU7UUUUU>U5U&U U?U U.U SD9n@UReU=UU6U@5unAnBURgU=U@UA5nAU=Ri5 U=RkU)U*5 U=Rm5(a4[oU=SE5(a#U"(dU=Rp"U!UUU40U.DSU'0DUD6 Ub6[s[uUR3555S;:a[wU=U5UUWBU5 U5bU5U=l<U5R{U=5 U0bUbUUSF'U=R}U0U W@USG9nAU(aU=WAR54$U=$!,(df  GNT=f!,(df  GN=f)Ha: Instantiate a pretrained pytorch model from a pre-trained model configuration. The model is set in evaluation mode by default using `model.eval()` (Dropout modules are deactivated). To train the model, you should first set it back in training mode with `model.train()`. The warning *Weights from XXX not initialized from pretrained model* means that the weights of XXX do not come pretrained with the rest of the model. It is up to you to train those weights with a downstream fine-tuning task. The warning *Weights from XXX not used in YYY* means that the layer XXX is not used by YYY, therefore those weights are discarded. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*): Can be either: - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co. - A path to a *directory* containing model weights saved using [`~PreTrainedModel.save_pretrained`], e.g., `./my_model_directory/`. - `None` if you are both providing the configuration and state dictionary (resp. with keyword arguments `config` and `state_dict`). model_args (sequence of positional arguments, *optional*): All remaining positional arguments will be passed to the underlying model's `__init__` method. config (`Union[PreTrainedConfig, str, os.PathLike]`, *optional*): Can be either: - an instance of a class derived from [`PreTrainedConfig`], - a string or path valid as input to [`~PreTrainedConfig.from_pretrained`]. Configuration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when: - The model is a model provided by the library (loaded with the *model id* string of a pretrained model). - The model was saved using [`~PreTrainedModel.save_pretrained`] and is reloaded by supplying the save directory. - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a configuration JSON file named *config.json* is found in the directory. state_dict (`dict[str, torch.Tensor]`, *optional*): A state dictionary to use instead of a state dictionary loaded from saved weights file. This option can be used if you want to create a model from a pretrained configuration but load your own weights. In this case though, you should check if using [`~PreTrainedModel.save_pretrained`] and [`~PreTrainedModel.from_pretrained`] is not a simpler option. cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. ignore_mismatched_sizes (`bool`, *optional*, defaults to `False`): Whether or not to raise an error if some of the weights from the checkpoint do not have the same size as the weights of the model (if for instance, you are instantiating a model with 10 labels from a checkpoint with 3 labels). force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. output_loading_info(`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. local_files_only(`bool`, *optional*, defaults to `False`): Whether or not to only look at local files (i.e., do not try to download the model). token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. attn_implementation (`str`, *optional*): The attention implementation to use in the model (if relevant). Can be any of - `"eager"` (manual implementation of the attention) - `"sdpa"` (using [`F.scaled_dot_product_attention`](https://pytorch.org/docs/master/generated/torch.nn.functional.scaled_dot_product_attention.html)) - `"flash_attention_2"` (using [Dao-AILab/flash-attention](https://github.com/Dao-AILab/flash-attention)) - `"flash_attention_3"` (using [Dao-AILab/flash-attention/hopper](https://github.com/Dao-AILab/flash-attention/tree/main/hopper)) - `"flash_attention_4"` (using [Dao-AILab/flash-attention/flash_attn/cute](https://github.com/Dao-AILab/flash-attention/tree/main/flash_attn/cute)). By default, if available, SDPA will be used. The default is otherwise the manual `"eager"` implementation. Accept HF kernel references in the form: /[@][:] - and are any non-"/" and non-":" sequences. - "@" is optional (branch, tag, or commit-ish), e.g. "@main", "@v1.2.0", "@abc123". - ":" is optional and selects a function inside the kernel repo. - Both options can appear together and in this order only: @revision first, then :kernel_name. - We intentionally allow a leading "|" prefix (e.g., "flash|...") because the code strips it before loading; '|' is not excluded in the character classes here. Examples that match: "org/model" "org/model@main" "org/model:custom_kernel" "org/model@v1.2.3:custom_kernel" experts_implementation (`str`, *optional*): The experts implementation to use in the model (if relevant). Can be any of: - `"eager"` (sequential implementation of the experts matrix multiplications). - `"batched_mm"` (using [`torch.bmm`](https://pytorch.org/docs/stable/generated/torch.bmm.html)). - `"grouped_mm"` (using [`torch.nn.functional.grouped_mm`](https://docs.pytorch.org/docs/main/generated/torch.nn.functional.grouped_mm.html)). By default, if the model supports it, `"grouped_mm"` will be used. The default is otherwise the manual `"eager"` implementation. > Parameters for big model inference dtype (`str` or `torch.dtype`, *optional*, defaults to `"auto"`): Override the default `torch_dtype` and load the model under a specific `dtype`. The different options are: 1. `torch.float16` or `torch.bfloat16` or `torch.float`: load in a specified `dtype`, ignoring the model's `config.dtype` if one exists. If not specified - the model will get loaded in `torch.float` (fp32). 2. `"auto"` - A `dtype` or `torch_dtype` entry in the `config.json` file of the model will be attempted to be used. If this entry isn't found then next check the `dtype` of the first weight in the checkpoint that's of a floating point type and use that as `dtype`. This will load the model using the `dtype` it was saved in at the end of the training. It can't be used as an indicator of how the model was trained. Since it could be trained in one of half precision dtypes, but saved in fp32. 3. A string that is a valid `torch.dtype`. E.g. "float32" loads the model in `torch.float32`, "float16" loads in `torch.float16` etc. For some models the `dtype` they were trained in is unknown - you may try to check the model's paper or reach out to the authors and ask them to add this information to the model's card and to insert the `dtype` or `torch_dtype` entry in `config.json` on the hub. device_map (`str` or `dict[str, Union[int, str, torch.device]]` or `int` or `torch.device`, *optional*): A map that specifies where each submodule should go. It doesn't need to be refined to each parameter/buffer name, once a given module name is inside, every submodule of it will be sent to the same device. If we only pass the device (*e.g.*, `"cpu"`, `"cuda:1"`, `"mps"`, or a GPU ordinal rank like `1`) on which the model will be allocated, the device map will map the entire model to this device. Passing `device_map = 0` means put the whole model on GPU 0. To have Accelerate compute the most optimized `device_map` automatically, set `device_map="auto"`. For more information about each option see [designing a device map](https://hf.co/docs/accelerate/main/en/usage_guides/big_modeling#designing-a-device-map). max_memory (`Dict`, *optional*): A dictionary device identifier to maximum memory if using `device_map`. Will default to the maximum memory available for each GPU and the available CPU RAM if unset. tp_plan (`Optional[Union[dict, str]]`, *optional*): A torch tensor parallel plan, see [here](https://pytorch.org/tutorials/intermediate/TP_tutorial.html). Use `tp_plan="auto"` to use the predefined plan based on the model. If it's a dict, then it should match between module names and desired layout. Note that if you use it, you should launch your script accordingly with `torchrun [args] script.py`. This will be much faster than using a `device_map`, but has limitations. tp_size (`str`, *optional*): A torch tensor parallel degree. If not provided would default to world size. device_mesh (`torch.distributed.DeviceMesh`, *optional*): A torch device mesh. If not provided would default to world size. Used only for tensor parallel for now. If provided, it has to contain dimension named `"tp"` in case it's > 1 dimensional, this dimension will be used for tensor parallelism offload_folder (`str` or `os.PathLike`, *optional*): If the `device_map` contains any value `"disk"`, the folder where we will offload weights. offload_buffers (`bool`, *optional*): Whether or not to offload the buffers with the model parameters. quantization_config (`Union[QuantizationConfigMixin,Dict]`, *optional*): A dictionary of configuration parameters or a QuantizationConfigMixin object for quantization (e.g bitsandbytes, gptq). subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. variant (`str`, *optional*): If specified load weights from `variant` filename, *e.g.* pytorch_model..bin. use_safetensors (`bool`, *optional*, defaults to `None`): Whether or not to use `safetensors` checkpoints. Defaults to `None`. If not specified and `safetensors` is not installed, it will be set to `False`. weights_only (`bool`, *optional*, defaults to `True`): Indicates whether unpickler should be restricted to loading only tensors, primitive types, dictionaries and any types added via torch.serialization.add_safe_globals(). When set to False, we can load wrapper tensor subclass weights. disable_mmap (`bool`, *optional*): Whether to disable memory mapping when loading safetensors checkpoints. When `None` (default), it is auto-detected to `True` when the checkpoint lives on an `hf-mount` FUSE filesystem (used by HF Spaces/Endpoints), where mmap + parallel page-faults can deadlock. When `True`, files are read fully into memory and parsed with `safetensors.torch.load`. When `False`, the default memory-mapped loader is always used. fusion_config (`dict[str, bool | dict[str, Any]]`, *optional*): Optional fusion configuration applied before model instantiation. Each key enables a fusion family and its value can either be `True` to enable that fusion with default options or a dictionary of family-specific options. For example, `{"patch_embeddings": True}` enables patch embedding fusion. This should only be used as an inference optimization, as it can slightly change outputs. If omitted, `from_pretrained()` falls back to `config.fusion_config` when available. Refer to the fusion mapping guide in `docs/source/en/fusion_mapping.md` for more details. key_mapping (`dict[str, str], *optional*): A potential mapping of the weight names if using a model on the Hub which is compatible to a Transformers architecture, but was not converted accordingly. kwargs (remaining dictionary of keyword arguments, *optional*): Can be used to update the configuration object (after it being loaded) and initiate the model (e.g., `output_attentions=True`). Behaves differently depending on whether a `config` is provided or automatically loaded: - If a configuration is provided with `config`, `**kwargs` will be directly passed to the underlying model's `__init__` method (we assume all relevant updates to the configuration have already been done) - If a configuration is not provided, `kwargs` will be first passed to the configuration class initialization function ([`~PreTrainedConfig.from_pretrained`]). Each key of `kwargs` that corresponds to a configuration attribute will be used to override said attribute with the supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute will be passed to the underlying model's `__init__` function. Activate the special ["offline-mode"](https://huggingface.co/transformers/installation.html#offline-mode) to use this method in a firewalled environment. Examples: ```python >>> from transformers import BertConfig, BertModel >>> # Download model and configuration from huggingface.co and cache. >>> model = BertModel.from_pretrained("google-bert/bert-base-uncased") >>> # Model was saved using *save_pretrained('./test/saved_model/')* (for example purposes, not runnable). >>> model = BertModel.from_pretrained("./test/saved_model/") >>> # Update configuration during loading. >>> model = BertModel.from_pretrained("google-bert/bert-base-uncased", output_attentions=True) >>> assert model.config.output_attentions == True ``` rNrroutput_loading_infoF_from_pipeline _from_autorrr max_memoryoffload_folderrrrrrrradapter_kwargs adapter_namerrgrrtp_sizerrtrust_remote_coderXrr key_mappingr)mirror _fast_initlow_cpu_mem_usagefrom_tf from_flaxoffload_state_dictT)rrrrrrrrzq`state_dict` cannot be passed together with a model name or a `gguf_file`. Use one of the two loading strategies. WORLD_SIZEraYou've set device_map=`auto` while triggering a distributed run with torchrun. This might lead to unexpected behavior. If your plan is to load the model on each device, you should set device_map={: PartialState().process_index} where PartialState comes from accelerate library)rrrzIaccelerate is required when loading a GGUF file `pip install accelerate`.repytorch) file_typer"from_auto_classusing_pipelinezN does not define `config_class`; pass an explicit config to `from_pretrained`.)return_unused_kwargsrrrrrzYou cannot combine Quantization and loading a model from a GGUF file, try again by making sure you did not passed a `quantization_config` or that you did not load a quantized model from the Hub.rgzxOne or more modules is configured to be mapped to disk. Disk offload is not supported for models loaded from GGUF files.zA kernel_config was provided but use_kernels is False; setting use_kernels=True automatically. To suppress this warning, explicitly set use_kernels to True.transformers_weights) rrrrrrrrrr)load_gguf_checkpointrr)return_tensors model_to_loadrrOr)register_fusion_patches)rerrrr)rrrrrrrrrrrrrrradjust_generation_fnr)r load_configr)@rTrsrrrrrrrrrGrdr@r3rmr!r)rfrom_pretraineddeepcopyrHr_rcrTrrrurrrrmodeling_gguf_pytorch_utilsrrrr]rfusion_mappingrrrr]rQpreprocess_modelrr"rrEr0r_load_pretrained_model_finalize_model_loadingevalrrerrrrar2rpostprocess_model load_adapterto_dict)CrOrrrrrrrrrrrr model_argsrrrrr from_pipelinerrrrrrrrrrrrrrrgrrrrrrrXrrrrKrXrdownload_kwargs_with_commit_adapter_model_pathr config_pathr) model_kwargsrrrrr dummy_modelrmodel_init_contextrerweight_conversionsr loading_infodisk_offload_indexsC rr PreTrainedModel.from_pretrainedsjZZ d3 **Y-ZZ d3 $jj)>F #3T:  **\59 7D)jj5 ZZ d3 ZZ d3 $4d; **%6>$jj)>EJJ{B/ jj6 **Y- **%5r:@bFFHzz.)< "JJ':DAJJ{D1 **Y-**Y-06 ;OQU0Vjj5 "JJ':DA"JJ':EBjj6  ?D9 jj5  )goGpD 4&Ap  "".EKE =E   %5# #, 0 " 'V&U-&U#  !'D'PT]TiD   C |S(I$J$J KKc   '"5/Lkj0 ,J W  )@)B)Bhi i  !NM` ) 'N N J:N .j9 #*Wfg  $+8J' (&"233$*$6&;GK]]6*F!L!&.+FK5@#M2 !F **0**5J*KF ' #v --3ZZ8P-QF *+; '\:, ( fj ' Y%J--&J  $ ? #C ? 11%GY[lmv& / 0<<|, ];> ^ rerr expected_keysc URnURnUSL=(a7 URR[R [R 1;nUc'[UR5R55OUn[R[R:a[U[USS55 SnUR baSUR R#5;aC[%UUR&UUR UR(UR*UR,5nUR b3U(d,[/UR U5n [1X UR5 /n [35(atU(dmUc<0n UH2n U R5[7U SUR8UR:S95 M4 U n[=XU5up[?U U [A5[A50S9nX4$[A5nUbUn GOUbUSRCS5(aUc0n UHnUR:(d[EU5(a?[GUS 5nU R5[IURK555 SSS5 Mc[MUS SS 9nUROU5 UR5HnURQU5U U'M M O?VZVlVl"!((01! )G cs-D>EE4'Ec[U[5(d URnSSKJs Jn [ X!5(d[US35eXlg)a% Register this class with a given auto class. This should only be used for custom models as the ones in the library are already mapped with an auto class. Args: auto_class (`str` or `type`, *optional*, defaults to `"AutoModel"`): The auto class to register this new model with. rNz is not a valid auto class.) rmrrtransformers.models.automodelsrrrr})rO auto_class auto_modules rregister_for_auto_class'PreTrainedModel.register_for_auto_classcsE*c**#,,J66{// |+FGH H$rc [U5(agUcURRcgURRUSS2SS/4;Ga#Sn[URSS5nURRb.URRURR:XdaURR b.URR URR:XdUbgX@RR:XaNUSURRSURRSURR S US 3 - n[ RU5 gg) zf Shows a one-time warning if the input_ids appear to contain padding and no attention mask was given. Nr<rzWe strongly recommend passing in an `attention_mask` since your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. sep_token_idz5 You may ignore this warning if your `pad_token_id` (z&) is identical to the `bos_token_id` (z), `eos_token_id` (z), or the `sep_token_id` (z ), and your input is not padded.)rvr pad_token_idrH bos_token_id eos_token_idrr)rr-r warn_stringrKs r%warn_if_padding_and_no_attention_mask5PreTrainedModel.warn_if_padding_and_no_attention_maskysK i   &DKK,D,D,L  ;; # #yRG'< <F #4;;EL))5$++:R:RVZVaVaVnVn:nKK,,8T[[=U=UY]YdYdYqYq=q ,AYAY1YLT[[MeMeLfg..2kk.F.F-GGZ[_[f[f[s[sZtu..:^;[]    ,- =rcUR(agURR(agURR(agg)z: Returns whether the model has a tensor parallelism plan. TF)r=rrrtrs rsupports_tp_plan PreTrainedModel.supports_tp_plans1 == ?? # # ;; ) )rcUR$)z0 Returns the model's tensor parallelism degree. )rtrs rrPreTrainedModel.tp_sizes }}rcUR(agURR(agURR(aggr)r>rrrrrs rsupports_pp_plan PreTrainedModel.supports_pp_plans1 == ?? # # ;; ) )rc[US5(a UR$[USS5nUb U[;a[R SUS35 Sn[U$)N_loss_functionriz `loss_type=zZ` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`. ForCausalLM)rr[rHrJrr)rris r loss_functionPreTrainedModel.loss_functionsh 4) * *&& &D+t4    =   i[)=> &II&&rcXlgr)r[rrs rr]r^s#rcr[5(d [S5eSSKJnJnJn SnSnUR U5 UR(d UROUc UROUnU"X"URRS9US9 S Ul UR U5 g!UR U5 f=f) zYTemporarily register hidden kernel wrappers so `kernelize` can discover and replace them.z`Kernels are not available. To use kernels, please install kernels using `pip install -U kernels`r)DeviceMode kernelizec[US05R5H5upU[UR55;dM$UR X5 M7 gN_hidden_kernels)rHr)rrwregister_module)rCrKrs rattach_hidden_kernels8PreTrainedModel.kernelize..attach_hidden_kernelssF#F,=rBHHJtF$9$9$;<<**44KrcB[US05Hn[X5 M grf)rHr)rCrKs rdetach_hidden_kernels8PreTrainedModel.kernelize..detach_hidden_kernelss(92>%?rN)ro)rmodeT) rgrrrbrcrdrFtraining INFERENCETRAININGrro _use_kernels)rrnrbrcrdrirls rrdPreTrainedModel.kernelizes#%%r  43 5  & . JJ, -)-4>>T\DMM_cD d6t{{/?/?#@t L $D  JJ, -DJJ, -s A%B##B6c[USS5$)NrrFrrs rrPreTrainedModel.use_kernelsst^U33rrc[U5(a[USS5(agU(aUR5 g[USS5(a[R S5 SUlg)NrrFzmDisabling kernels at runtime is a no-op as there is no 'unkernelize' routine; keeping current kernels active.)rrHrdrrrrr`s rrrusX ;;74??   NN t^U33##D!&D rcompile_configcSURR;a UR$U=(d [5n[ UR SS5=(d [5n[ US5(a[ USU5U:wa:Xl[R"UR40UR5D6Ul UR$)atReturn a `torch.compile`'d version of `self.__call__`. This is useful to dynamically choose between non-compiled/compiled `forward` during inference, especially to switch between prefill (where we don't want to use compiled version to avoid recomputing the graph with new shapes) and iterative decoding (where we want the speed-ups of compiled version with static shapes).llama4rwN_compiled_call_last_compile_config) r model_type__call__r)rHrgrr{rrrrz)rrwdefault_configs rget_compiled_call!PreTrainedModel.get_compiled_calls t{{-- -== ':=? !7!79I4PcTaTc.//t3^DV(6 %"'-- "ZAWAWAY"ZD """rcUR$r)rArOs ris_backend_compatible%PreTrainedModel.is_backend_compatibles...rrrrrc USLn[5(aU(dg[5(a[5(d|U(duUR5H&upg[R "USS9n[ XU5 M( UR5H&upi[R "U SS9n[ XU5 M( gXRR5- HanURU5n[X&SS9n [R"XzS9nUb [XXvSSUR5U5 MU[ XU5 Mc UR5H/upi[X&SS9n [R"XS9n[ XU5 M1 g)aMove the missing keys (keys that are part of the model parameters, but were NOT found in the loaded state dicts) back from meta device to their device according to the `device_map` if any, else cpu. Takes care of sharding those missing parameters if `device_mesh` is provided, i.e. we are using TP. All non-persistent buffers are also moved back to the correct device (they are not part of the state_dict, but are not missing either). NrrT)valid_torch_deviceF)r.r/rrr zeros_likerrrqr+rr5 empty_likerHget_local_ranknamed_non_persistent_buffers) rrrrrrr rrr param_device buffer_devices rr46PreTrainedModel._move_missing_keys_from_meta_to_devicesT$4/ % ' '    %9%;%;L"335 ((u=*4e<6 $113 ((>*4e< 4   "<"<"A"A"CCC005E%j$OL$$U@E&+T5+:T:T:VXc +4e<D <<>KC&z4PM$$VBE &t% 8?rc N[5(aF[5(d7UR5HnURU5nSUlM SUl[ 5(aU(dSSKn[URSS9R5Vs1sHn[USS5(aMUiM sn5nURRUSS9 UR5 SSS5 gUR5 g![ a Mf=fs snf!,(df  g=f)a Initialize the missing keys (keys that are part of the model parameters, but were NOT found in the loaded state dicts), according to `_initialize_weights`. Indeed, since the corresponding weights are missing from the state dict, they will not be replaced and need to be initialized correctly (i.e. weight initialization distribution). Also marks non-missing params/buffers with `_is_hf_initialized` and propagates this flag to modules, so that `_initialize_weights` can skip fully-initialized modules entirely. TrN) keep_varsrxFr)r/rrrrxAttributeErrorr.rrrrHrrr)rrr param_or_bufferrr5not_initialized_parameterss rr6(PreTrainedModel._initialize_missing_keysKs   %9%;%; (&*&B&B3&GO9=O6) '+D # & ' ' *. OOdO;BBDtDqGTUWkmrLsDt* &223M]^2_'')`_  # # %&u__s)DD.DD D D D$c[SUR555nU(aS/O/n[SUR555nU(aURS5 UR=(d /nUR=(d /U-nSupx[ U5S:a,[ R"SRSU555n[ U5S:a,[ R"SRS U555nUb5URV s1sHoRU 5bMU iM sn Ul Ub6URV s1sHoRU 5bMU iM sn Ul g g s sn fs sn f) zAdjust the `missing_keys` and `unexpected_keys` based on current model's exception rules, to avoid raising unneeded warnings/errors. This is performed in-place. c3H# UHupURS5v M g7f)zrotary_emb.inv_freqNr&rrrXs rrFPreTrainedModel._adjust_missing_and_unexpected_keys..vs!"p[oif6??3H#I#I[o "zrotary_emb\.inv_freqc3H# UHupURS5v M g7f) position_idsNrrs rrrys &mXl96v~'F'FXlrz(^|\.)position_ids$)NNrrZc3.# UH nSUS3v M g7frYrNrrpatterns rrrs6gVf7!G9AVfrc3.# UH nSUS3v M g7frrrs rrrs9mYlgQwiq/YlrN) rqrrQr6r7rrirrrrjr#) rrhas_inv_freq_buffersadditional_unexpected_patternshas_position_ids_buffersmissing_patternsunexpected_patternsignore_missing_regexignore_unexpected_regexr s rr73PreTrainedModel._adjust_missing_and_unexpected_keysosc #"p[_[m[m[o"ppFZ*A)B`b&#&&mX\XjXjXl&m#m # * 1 12H I??E2#FFL"Pnn8B5  1 $#%::chh6gVf6g.g#h " #a '&(jj9mYl9m1m&n # ++88)86E>F.Fcb[US05R5H!nURU5n[USS5 M# UR 5(aUUR Vs1sH7nX@R ;d#[URU5SS5(aM5UiM9 snUlggs snf)a-Adds the `_is_hf_initialized` flag on parameters that will be tied, in order to avoid initializing them later as they will be tied (overwritten) anyway. This is very important as most embeddings are tied, and they are huge params (vocabularies are often 256k), so running inits on them is very costly.rqrxTFN)rHr+rprrrrqr)rr tied_paramrr s rr30PreTrainedModel.mark_tied_weights_as_initializeds "$(?DIIKJ&&z2E E/ 6L    (44)4C444t;;C@BVX]^4)L % !)s $4B,B,rcURU5$![a Of=fURU5$![a Of=f[X5up#US:Xan[ UR S[ RRR5[ RRRLaUR5$[SUS35e)aI Return the parameter or buffer given by `target` if it exists, otherwise throw an error. This combines `get_parameter()` and `get_buffer()` in a single handy function. If the target is an `_extra_state` attribute, it will return the extra state provided by the module. Note that it only work if `target` is a leaf of the model. _extra_stateget_extra_state`z2` is neither a parameter, buffer, nor extra state.) rpr get_bufferrUrHrrrr'r)rrrCrs rr'PreTrainedModel.get_parameter_or_buffers  %%f- -    ??6* *   1$? . (((*;UXX__=\=\]88??223))+ +q(Z[\\s  5 AArrc## URXS9HJup4SU;aURSS5OSU4upVURU5nXeR;dMEX44v ML g7f)zSimilar to `named_buffers`, but only yield non-persistent ones. It is handy as it's not perfectly straightforward to know if they are persistent or not)rrrFrrN)rrr_non_persistent_buffers_set)rrrrKrrbuf_names rr,PreTrainedModel.named_non_persistent_bufferssg !..w.bLD7:Tkt{{32Dz F''/F===l" cs AA! A!rnch>[TU]U5nUR(aUR5 U$r)rJtrainrrd)rrnoutrs rrPreTrainedModel.trains)gmD!    NN  rc$URS5$)NF)rrs rrPreTrainedModel.evalszz%  rcURSL$r)r}rs rrPreTrainedModel.is_remote_codesd**r)rzrprMrxr3r4r{r[r1r>r?r>r=rrrqrrgrrirer,r]rrr)rNFr r7)NNT)NFT)TNF50GBNNTT)T) AutoModel)TT)rrrrrr)ror!rr}r*rr+rr,rr.r0r1rar2r3r4rErr5r6r7r8r9r:r;r<r=rtr>r r?r@rArBrrcompilerallow_in_graphrzrDrrGrKr[rrrsetterrrzr classmethodrrr'rrerrrrrrrr^rbrrr&rr0r3rErJr^rdrkrnno_gradrrrguard_torch_init_functionsrrvrrrrrrrrrr r rr5r8ryrrrErQrWrPathLiker~r r_r[rrr*rrrrrr^rrrr r rrwrrrArHrPrSrrXr]rdrr)rrrSr4r6r7r3rrrrrrr __classcell__rs@rrrs (37L$'(4/6KsL$#'JS D ''OS&)/cDIo.6:s3x$s)+d29:>tCy4!7> :>3s8d3i/$6=@D #c(T#Y"6"=D*.S#X-57"DcN78<#T#Y%5<;?&S D(8?04T#Y-4!ND !&$& %%:>%tCy4'7> $Hd38n#H,0Hd3c3h'(/-2#T1#(D().-'++  ^^""'.Dn)<$='.#'.R9d3 #4599 /0&M/&MP/>bc3hc5c?23 ^^!DcNT1!!F ^^DeCHo!56= :; ,49s?,t,@EEN;BII;; #T##VFH-1@@%-@!) @ !x}!5s!:; @ #(hm(Bl._cl. l.\ 1sUYz 1^a 1$$3:$$^b$$or$$L"C$J"SV"0T$ 5 5 5d83:d8Z^d8L*Wt*WX*X )C$J@%S4Z%4.%" ]]_7?7?r)$)2 ]]_ $$&H'H8p%Dp%Tp%dQ8C4Q8SWQ8f M&*)-" 9d 9 $J9 9  9v%+T&*)-" ] ]d ] $J ]  ]  ]D&* " AYYAd A A  A  AF@!@@,d d c  bll8K)K 2(.T:>gq$\d,/(n4nn!%"&!$*"#'!%%)\bkk)\\4K \  \ c \t\TzD \\#\| > % %& 4' 4$ 588??   -!-2 588??  :+:+x'(KK/3IMbfimbm>U[[T %%,:M%%N ?C.2(-$!&#''+!AE$(M - .M'*R[['84'?M!3&4t; M $t+ M "& MMMTzD MMMMCS#X!667$>MTkM %!MM^ +/ c0 c04Kc0s)d*c0) c0 Cy4' c0  $& ' c0c0J$/$?P$ $$L!.%%*#-J     ' '$$.64T44 & &$ & &# 0D##$///93i/94K/9- /9 "D( /9  /9b"&T"&d"&H @Q VZ D8]c]2>B # #6: # %U\\)* + #$ !+t++rrz model file)object object_class object_files recursivecgrrrers rrzrzsVYrcgrrrs rrzrzsJMrc[5(a0nU(aXS'[U40UD6$[US5(a[UR5$U$)a Recursively unwraps a model from potential containers (as used in distributed training). Args: model (`torch.nn.Module`): The model to unwrap. recursive (`bool`, *optional*, defaults to `False`): Whether to recursively extract all cases of `module.module` from `model` as well as unwrap child sublayers recursively, not just the top-level distributed containers. rrC)rdr}rrzrC)rerrs rrzrzsP   "+; *5;F;; 5( # # - -LrrcVUS:Xag[R"U5RS;$)zCheck if the device is an accelerator. We need to function, as device_map can be "disk" as well, which is not a proper `torch.device`. rgF)rr)rrrors ris_accelerator_devicers)||F#((??raccelerator_device_mapc[S5nURR5n[5(a URO/nUR 5HupgXd;aM UR U5nUbURXU5n OUR5n UR5U -n [U5S:a![XeSS9SLn X(a [5OS-n X7==U - ss'M U$)z This utility function calculates the total bytes count needed to load the model on each device. This is useful for caching_allocator_warmup as we want to know how much cache we need to pre-allocate. cg)Nrrrrrr&get_total_byte_count..s1rNrT) is_weightr) rrqr+rr=r)rparam_element_sizer@rrrDr) rerrtotal_byte_counttied_param_namesrrrr dtype_sizeparam_byte_countis_part_of_plans rget_total_byte_countrs#9-22779 A C CennG4::<  ) --j9  #%88ERJ++-J ;;=:5 w 1) as cudaMalloc is not a bottleneck at all anymore - Loading speed bottleneck is now almost only tensor copy (i.e. changing the dtype) and moving the tensors to the devices. However, we cannot really improve on those aspects obviously, as the data needs to be moved/copied in the end. N)rxpugArrg333333ArF)rrr)r)rrrrrorHrcurrent_device mem_get_infomemory_reservedmemory_allocatedrr/rr)rer+rrrrr byte_countaccelerator_modulerfree_device_memorytotal_device_memory unused_memoryrXs rr(r(4sg*:M9R9R9T9T XmntXu#u||F##9T "+E<X/446 ;;/ )!( !< $*LL$>uEHZHkHklqHrrM)M9'7 +m;!1$'99!"J"/!2J Z})LMJ KKJ!O,EMM&`e fG7s E+E+c V^\rSrSrSr\\\\\\\\\ \ S. r S\ S\ S\ 4U4SjjrSrU=r$) AttentionInterfaceiwaO Dict-like object keeping track of allowed attention functions. You can easily add a new attention function with a call to `register()`. If a model needs to locally overwrite an existing attention function, say `sdpa`, it needs to declare a new instance of this class inside the `modeling_.py`, and declare it on that instance. ) flash_attention_4flash_attention_3rr rzpaged|flash_attention_4zpaged|flash_attention_3zpaged|flash_attention_2z paged|sdpaz paged|eagerrrrc>Uc[RS5 OUS:waX;a[SUS35e[TU]X5$)zcReturn the requested `attn_implementation`. Also strictly check its validity, and raise if invalid.a You tried to access the `AttentionInterface` with a `config._attn_implementation` set to `None`. This is expected if you use an Attention Module as a standalone Module. If this is not the case, something went wrong with the dispatch of `config._attn_implementation`rrzP` is not a valid attention implementation registered in the `AttentionInterface`)rrKeyErrorrJr)rrrrs r get_interface AttentionInterface.get_interfaces[  &   K  !G +0C0O'((xy w{.88rr)rrrrrr:r<rAr;rBr8_global_mappingrrrrrrs@rrrwsN5440&#:#:#:24 O 9 9x 9H 9 9rrr cp\rSrSrSr\S\R4Sj5r\S\R4Sj5r Sr g) PreTrainedAudioTokenizerBaseia Class that additionally defines the behavior of any `audio_tokenizer` to be added. Characteristic for any of them: 1. Encode raw audio into discrete audio codebooks (with x channels) 2. Decode from discrete audio codebooks back to raw audio It is possible that they can decode in different ways given a different representation but they are forced to support 2. nonetheless, e.g. see `DAC`. input_valuescg)zq Encode raw audio retrieved from a respective `FeatureExtractor` into discrete audio codebooks (with x channels) Nr)rrrrs rencode#PreTrainedAudioTokenizerBase.encoder audio_codescg)z6Decode from discrete audio codebooks back to raw audioNr)rrrrs rdecode#PreTrainedAudioTokenizerBase.decoderrrN) rrrrrrrrrrrrrrrrsH 5<<  E%,,EErrr)rTN)NNNr(r`rsrCrLrrrir rabcrrcollections.abcrr contextlibr dataclassesrr r r itertoolsr threadingr typingrrrrrzipfilerrhuggingface_hubrrr packagingr safetensorsrsafetensors.torchrr'rrrrtorch.distributionsrtorch.utils.checkpointrrr rconfiguration_utilsr!conversion_mappingr"core_model_loadingr#r$r%r&rr'dynamic_module_utilsr( generationr)r* integrationsr+r,r-r.r/integrations.accelerater0r1r2r3r4r5r6rr7integrations.eager_pagedr8integrations.finegrained_fp8r9integrations.flash_attentionr:integrations.flash_pagedr;integrations.flex_attentionr<rr=r>integrations.moer?integrations.peftr@integrations.sdpa_attentionrAintegrations.sdpa_pagedrBintegrations.tensor_parallelrCrDrErFrGrHrIloss.loss_utilsrJmodeling_flash_attention_utilsrKrLrMrNmodeling_rope_utilsrOmonkey_patchingrPrQ pytorch_utilsrR quantizersrSquantizers.autorTquantizers.quantizers_utilsrUsafetensors_conversionrVutilsrWrXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirjrk utils.genericrlrmrn utils.hubrorprqutils.import_utilsrrrsrtrurvutils.loading_reportrwrxutils.output_capturingryrzutils.quantization_configr{accelerate.hooksr|accelerate.utilsr}_typingr~ is_availabler!smdistributed.modelparallel.torch modelparallelrsmdistributed.modelparallelr SMP_VERSIONrr get_loggerrrrrupperrrrrrrrrrrrrrrrrrruint8int8int16uint16rrint32uint32rfloat64int64uint64 float8_e4m3fn float8_e5m2r.rrrrr:rBr'rrNrar r^rdrrrrorrrrrr[rrdrzrrr(rr rrrrrrCs  #.%($HH \\!69+-$1< +47vvFCCA=?F32?A* 5B,#-=3.ji\\KH93<' %00==?33F ' k :gmmF>S S %   H %zz~~nc288: JJNN#6<BBD%&CK\]  $---64.3. `##0U[[0C$J00.  1 JJ ++ ** ;; << == NN ;; << == == ;; <<""  &-$6(- $ /k2;;&/k $/k/k+ /k  #u||  /kdU\\c"))S ,DSN,S%,,=N8O,TYZ^_bcf_gZhjnorjsZsTt,>% #c(^%)-c5<<.?)@% 4C>4C> )*%*MS%,,&'M0AM #u|| M`(&7(S(RWR^R^(sS4Z326-1"I.#&#4t#;I. 4ZI.TzI.D[ I. t I.  I.%($JI.$d*I.t I. 49t TD[ ()I.f(,P  t #d *P3i$&P PTk P t P  P$P U[[ ()Pf[[|Z*Z*z^9+bii!57GYi^9+Bs((C(CD&&2*9*E*E*M*M*T*T[|+U+O'  YYDY_Y Y M MdMryyM M dryy2@#)ell":@t@^b 48HSVZHZD@gO@g$@g^ilp^p@gF"9)"9L/A.B+BE?Er