ZjSrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSK r SSKJr SSKJr SSKJr SSKJrJr SSKJr SS KJrJrJrJrJr SS KJ r J!r! \RD"\#5r$S \%S \%4S jr&\ RN"5r(Sr)S \%\RT-S S4Sjr+S\%\RT-S \,\%4Sjr-S\%\RT-S \,\%4Sjr.S\%\RT-S \,\%4Sjr/S\%\RT-S \,\%4Sjr0SS.S\%S\%\RT-S\1S \24Sjjr3S\%\RT-S\%\RT-S\%\RT-S\,\%S \%4 Sjr4S4S\%\RT-S\%S \%\RT-S-S!\1S"\5\%\%4S-S#\1\%-S-S$\%S-S%\1S&\%S-S'\%S-S \%4S(jjr6S4S)\%S\%\RT-S \%\RT-S-S!\1S"\5\%\%4S-S#\1\%-S-S$\%S-S%\1S&\%S-S*\%S-S \24S+jjr7S5S,\S-\%\RT-S.\5S-S \,\%4S/jjr8S0r9S1r:S6S2jr;S7S3jr""xj dV[U[R5(ag[U[R5(aURn[R "U5Hn[U[R 5(dM$[URSS5nURS5(aURS5(d'[T RRU5(dM g O[U[R5(aLURH;nUR R#S5SnU(dM*TR%U5 M= Ov[U[R&5(aWUR(S:XaGUR*(a6UR*R#S5SnU(aTR%U5 [R,"U5H nT"U5 M g)Nid available is_flash_attnrr) isinstanceastTryIftestwalkCallgetattrfuncendswith startswithhasattrutils import_utilsImportnamesrsplitadd ImportFromlevelmoduleiter_child_nodes) noderacondition_nodecheck_functionalias top_modulechildimported_modulesrecursive_look_for_imports transformerss r!rz/get_imports..recursive_look_for_importssb dCGG $ $  cff % %99D"%((4.nchh77%,^-@-@$%KN&// <<*55oFF"<#5#5#B#BNSS#1cjj ) )"ZZ--c215 :$((4$cnn - -zzQ4;;![[..s3A6 $((4))$/E &u -0r#)r>r?rDtransformers.utilsr^parsesorted)rVrErFtreeryrzr{s @@@r! get_importsrs` h )Q&&( *u.B 99W Dt$ " ##U * )s A&& A4c[U5n/nUHn[R"U5 M [U5S:a0[SSRU5SS RU5S 35e[U5$![aGn[R SUSU35 S[ U5;aURU5 SnAMeSnAff=f) a Check if the current Python environment contains all the libraries that are imported in a file. Will raise if a library is missing. Args: filename (`str` or `os.PathLike`): The module file to check. Returns: `list[str]`: The list of relative imports in the file. z&Encountered exception while importing z: zNo module namedNrz\This modeling file requires the following packages that were not found in your environment: z, z. Run `pip install  `) rr/ import_module ImportErrorrrrLr*rMjoinrH)rVimportsmissing_packagesimp exceptions r! check_importsrs(#G   # #C ( q  jyy)*++>sxxHX?Y>ZZ[ ]   ))!  NNCC59+V W!C N2 '',, sA88 C ;CCC F force_reload class_namerRrc [RRU5nURS5nUR [RR S5n[ [5U- n[ U(a5[RRUS5 [R"5 [RRU5n[RR!X4S9nU/[#[%[ ['U555-n[(R*"SR-SU555R/5nUc3[RR1U5n U [RU'OUn [3U SS5U:wa!UR4R7U 5 Xl[3X5sSSS5 $!,(df  g=f) a Import a module on the cache directory for modules and extract a class from it. Args: class_name (`str`): The name of the class to import. module_path (`str` or `os.PathLike`): The path to the module to import. force_reload (`bool`, *optional*, defaults to `False`): Whether to reload the dynamic module from file if it already exists in `sys.modules`. Otherwise, the module is only reloaded if the file has changed. Returns: `typing.Type`: The class looked for. rJrN)locationr#c3Z# UH!n[U5UR5-v M# g7fN)bytes read_bytes).0rEs r! &get_class_in_module..*s"2cVbQR58alln3LVbs)+__transformers_module_hash__rZ)r+r)normpath removesuffixrseprr _HF_REMOTE_CODE_LOCKr(modulespopr/r0getutilspec_from_file_locationrmaprUhashlibsha256r hexdigestmodule_from_specrdloader exec_moduler) rrRrrr8 cached_module module_spec module_files module_hashrqs r!get_class_in_moduler sT& 77  K (D   U #D << S )D-.>#((2cVb2c*cdnnp   ^^44[AF &CKK "F 692 >+ M    * *6 22= /v*-   s 3E G  Gpretrained_model_name_or_pathresolved_module_filemodules_neededc<^ [U5R5m [U5Rn[U5R5nS[S[4U 4SjjnU"U5U4/nUH5nT U- US3- R5nUR U"U5U45 M7 [ R "5n [USS9HDupU RU RS55 U RU R55 MF U R5SS $) zm Computes a stable hash from the bytes of the local source file and its direct relative-import source files. source_file_pathrc>URT5R5$![a UR5s$f=fr) relative_toas_posix ValueError)r model_paths r!_resolve_relative_source_pathG_compute_local_source_files_hash.._resolve_relative_source_pathHs@ /#// ;DDF F /#,,. . /s "??rJc US$)Nr)entrys r!2_compute_local_source_files_hash..Ws ERSHr#)keyr:N) rr4r5rLr*rrrupdateencoderr) rr8rr module_parentr files_to_hash module_neededmodule_needed_pathsource_files_hash relative_path file_pathrs @r! _compute_local_source_files_hashr:s34<<>J%,,M 45==?/// ''; <>RSM( (=8m_C;PPYY[;T$U   !5!5g!>?  !5!5!78%V  & & (" --r# cache_dirforce_downloadproxiestokenrevisionlocal_files_only repo_type _commit_hashc [5(aU(d[RS5 Sn[U5n[R R U5n Sn U (dQ[R RR[[URS555n [XX)US9n /n[UUUUUUUUUU S9 nU (dX:waURU5 [!U5nU (a[[R R#[R R%U555n['XUU5nU(a,[R RRUU/5n OUn [([R R-W -n[+U5 [-[.5U- nU (GaCUU- R15(a([2R4"U[UU- 55(dLUU- R6R9SSS 9 [:R<"UUU- 5 [>R@"5 UHn[-U5R6US 3- n[R RUU5nUU- R15(a*[2R4"U[UU- 55(aM[:R<"UUU- 5 [>R@"5 M GO&[CX5nUU- nU[R R-U-n[R RUU5n[+[-U5R65 UU- R15(d/[:R<"UUU- 5 [>R@"5 UHknUU- R6US 3- R15(aM-[EU[-U5R6U- S 3UUUUUUUS 9 URUS 35 Mm [GU5S :aUUcRSRUVs/sHnSU3PM sn5nUcSOUS3nSUU3n[RISUSUS35 [R RUU5$![a [RSUSUS 35 ef=fs snf)a Prepares Downloads a module from a local folder or a distant repo and returns its path inside the cached Transformers module. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained model configuration hosted inside a model repo on huggingface.co. - a path to a *directory* containing a configuration file saved using the [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`. module_file (`str`): The name of the module file containing the class to look for. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. repo_type (`str`, *optional*): Specify the repo type (useful when downloading from a space for instance). Passing `token=True` is required when you want to use a private model. Returns: `str`: The path to the module inside the cache. z+Offline mode: forcing local_files_only=TrueTN/)rrr)rrrrrrrrzCould not locate the z inside r)parentsr&rJ)rrrrrrrr z- rZzs/zhttps://huggingface.co/z9A new version of the following files was downloaded from z: z . Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.)%rrinforLr+r)isdirrrrr"rmrr r*OSErrorrbasenamerrr r6rr r-filecmpcmpr5mkdirshutilcopyfiler/r0r get_cached_module_filerMr)rr8rrrrrrrrdeprecated_kwargsis_localr submodule new_filesrrlocal_model_namelocal_source_files_hashfull_submodulesubmodule_pathrmodule_needed_file commit_hashfull_submodule_module_file_pathrE repo_type_strurls r!rr^skt!1 AB%((E$F!ww}}:;HM GGKK$$S)>@]@c@cdg@h%ij . ))nw I* ) )-% MA   [ )##78N01A1A"''BRBRSpBq1rs"B )8Ln#   ((*:M (+5'"''++5 C*,'',,~{*S'd#BCJJK,4466 OO0.;2N O  ' ' )+M#k199}oS Calling this function will execute the code in the module file found locally or downloaded from the Hub. It should therefore only be called on trusted repos. Args: class_reference (`str`): The full name of the class to load, including its module and optionally its repo. pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained model configuration hosted inside a model repo on huggingface.co. - a path to a *directory* containing a configuration file saved using the [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`. This is used when `class_reference` does not specify another repo. module_file (`str`): The name of the module file containing the class to look for. class_name (`str`): The name of the class to import in the module. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. repo_type (`str`, *optional*): Specify the repo type (useful when downloading from a space for instance). code_revision (`str`, *optional*, defaults to `"main"`): The specific revision to use for the code on the Hub, if the code leaves in a different repository than the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. Passing `token=True` is required when you want to use a private model. Returns: `typing.Type`: The class, dynamically imported from the module. Examples: ```python # Download module `modeling.py` from huggingface.co and cache then extract the class `MyBertModel` from this # module. cls = get_class_from_dynamic_module("modeling.MyBertModel", "sgugger/my-bert-model") # Download module `modeling.py` from a given repo and cache then extract the class `MyBertModel` from this # module. cls = get_class_from_dynamic_module("sgugger/my-bert-model--modeling.MyBertModel", "sgugger/another-bert-model") ```z--rrJ)rrrrrrrr)rmrr)rrrrrrrrrrkwargsrepo_idr8r final_modules r!get_class_from_dynamic_modulersn #2#8#8#> /-33C8K!>!I )e%) L zn UUr#objfolderconfigct^TRS:Xa[RSTSUS35 gU4Sjn[U[[ 45(aUH nU"U5 M O UbU"U5 /n[ RTRRn[U5[U5R- n[R"Xg5 URU5 [U5HKn[U5[U5R- n[R"X5 URU5 MM U$)a Save the modeling files corresponding to a custom model/configuration/tokenizer etc. in a given folder. Optionally adds the proper fields in a config. Args: obj (`Any`): The object for which to save the module files. folder (`str` or `os.PathLike`): The folder where to save. config (`PreTrainedConfig` or dictionary, `optional`): A config in which to register the auto_map corresponding to this custom object. Returns: `list[str]`: The list of files saved. __main__z We can't save the code defining z in z as it's been defined in __main__. You should put this code in a separate module so we can include it in the saved folder and make it easier to share via the Hub.Nc>T RRnURS5SnUST RR3nSU;aSnSnT RRR S5(agUST RR3n[ T SS5b=[ T S5nURnURS5SnUSUR3nOUST RR3nXE4n[ U[5(a%URS05n X9T R'XS'g[ USS5bX0RT R'gT RU0Ul g)Nr TokenizerFastslow_tokenizer_classauto_map) __class__ __module__rm__name__rfrdr]dictr _auto_classr) _config module_name last_module full_namerfast_tokenizer_classslow_tokenizerslow_tok_module_namelast_slow_tok_modulerrs r!_set_auto_map_in_config3custom_object_save.._set_auto_map_in_configsomm.. !'',R0 "m1S]]%;%;$<= ) ##' #' }}%%..v66*5a 8N8N7O'P$3 6=I%,S2H%IN+9+D+D(+?+E+Ec+J2+N(.B-C1^E\E\D]+^(+6a 8N8N7O'P$-DI gt $ ${{:r2H(1S__ %"*J  Wj$ / ;09  S__ - #;G r#)rrrr]rCtupler(r__file__rrrrr*rU) rrrrcfgresult object_file dest_file needed_files ` r!custom_object_savervs  ~~#.se4x@  <@&4-((C #C (  ' F++cnn-66KV [ 1 6 67I OOK+ MM)1= LD$5$:$:;  / i > Mr#c[S5e)NzLoading this model requires you to execute custom code contained in the model repository on your local machine. Please set the option `trust_remote_code=True` to permit loading of this model.)r)signumframes r!_raise_timeout_errorrs  c r#cUcdUb SUSUSUS3nOT[RRU5(a'SUS[RRU5S3nO SUSUS3nUGcU(aSnGOU(a[S :aSn[ R "[ R [5n[ R"[5 UcC[US US 35nUR5S ;aS nOUR5S;aSnUcMC[ R"S 5 Ub;[ R "[ R U5 [ R"S 5 OU(a [SS5 U(aU(dU(d[US US35eU$![a [US US35ef=f!Ub<[ R "[ R U5 [ R"S 5 ff=f)a Resolves the `trust_remote_code` argument. If there is remote code to be loaded, the user must opt-in to loading it. Args: trust_remote_code (`bool` or `None`): User-defined `trust_remote_code` value. model_name (`str`): The name of the model repository in huggingface.co. has_local_code (`bool`): Whether the model has local code. has_remote_code (`bool`): Whether the model has remote code. error_message (`str`, *optional*): Custom error message to display if there is remote code to load and the user didn't opt-in. If unset, the error message will be regarding loading a model with custom code. Returns: The resolved `trust_remote_code` value. NzThe repository z% references custom code contained in zm which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/z . zt contains custom code which must be executed to correctly load the model. You can inspect the repository content at z contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/Frz9 You can inspect the repository content at https://hf.co/z. You can avoid this prompt in future by passing the argument `trust_remote_code=True`. Do you wish to run the custom code? [y/N] )yesy1T)non0rZzS. Please pass the argument `trust_remote_code=True` to allow custom code to be run.) r+r)rabspathTIME_OUT_REMOTE_CODEsignalSIGALRMralarminputlower Exceptionr)trust_remote_code model_namehas_local_codehas_remote_code error_message upstream_repoprev_sig_handleranswers r!resolve_trust_remote_coder4s.  $!*-RS`Rab,,9?$@  WW]]: & &!*. ggooj9:$@ "*.,,6 readlinesstriprgrrr/metadatarPackageNotFoundErrorr*r from_stringvaluer~rrr) r6requirements_filerfailed requirementsrE requirement package_name delimiterversion_numberlocal_package_version is_satisfieds r!check_python_requirementsrJ!sF" "i?ibhi , $;;=L%(K%++-K+"8"8"="= 7:OP[:\7   (1(:(:(B(B<(P% $)C0<B?BSSW X[_[d[dek[l l  E% $ 7* ,6) > 7 %%::   -?@A     spFD=rZs:  CI   H %B!~~' & &bkk 1&d&,'cBKK&7'DI'* 3+< c @4$# +4$S 4$n *C"++- *$s) *N -+-+r{{"-+ -+  -+`!.#&#4!.r{{"!. +!.I !.  !.N+/ %)#" #g5#&#4g5g5R[[ 4'g5 g5 #s(^d " g5 #:  g5Djg5g5Tzg5*g5 g5Z+/ %)#" $kVkV#&#4kVR[[ 4'kV kV #s(^d " kV #:  kVDjkVkVTzkV:kV kV\ICIr{{):ID4KI[_`c[dIXgkRj5 r#