Zj-bSrSSKJr SSKJr SSKJr SSKJrJ r \ R"\ 5r \"SS9\"S S \555r \"SS9\"S S \555r\"SS9\"S S\555r\"SS9\"SS\555r\"SS9\"SS\555r/SQrg)zBlt model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstringloggingzitazap/blt-1b-hf) checkpointc>^\rSrSr%SrSrSrSr\\ S'Sr \ S-\ S 'S r \S-\ S 'S r \S-\ S 'Sr\\ S'Sr\\ S'Sr\S-\ S'Sr\\ S'Sr\\ S'Sr\\-S-\ S'Sr\\ S'Sr\\-S-\ S'Sr\\ S'Sr\S-\ S'Sr\\ S 'U4S!jrS"rU=r$)#BltLocalEncoderConfigF cross_attn_all_layers (`bool`, *optional*, defaults to `True`): Whether all attention layers have cross attention. cross_attn_k (`int`, *optional*, defaults to 2): Number of cross-attention heads used in the model. hidden_size_global (`int`, *int*, defaults to 2048): Hidden size of the global transformer layer. blt_local_encoderA vocab_sizeFNcross_attn_all_layers cross_attn_khidden_size_global hidden_sizenum_attention_headsnum_key_value_headsnum_hidden_layersh㈵> rms_norm_epsdropout`max_position_embeddingsrope_parameterssilu hidden_actintermediate_size{Gz?initializer_rangec >UR=(d URUlUR=(d [SUR-S- 5UlSUl[ TU]"S0UD6 gNrF)rrr'intrtie_word_embeddingssuper __post_init__selfkwargs __class__s z/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/blt/configuration_blt.pyr1#BltLocalEncoderConfig.__post_init__9s]#'#;#;#Wt?W?W !%!7!7!X3q4CSCS?SVW?W;X#(  '')r'rr/) __name__ __module__ __qualname____firstlineno____doc__ model_type default_thetarr.__annotations__rboolrrrrrrrfloatr!r#r$rdictr&strr'r)r1__static_attributes__ __classcell__r5s@r6r r s%JMJ).4$;. L#* %)d )K!!&*t*sL%"%GUS[4 %#(S(48O^d*T18J$(sTz(#u#((r8r c^\rSrSr%SrSrSrSr\\ S'Sr \ S-\ S 'S r \S-\ S 'S r \S-\ S 'Sr\\ S'Sr\\ S'Sr\S-\ S'Sr\\ S'Sr\\ S'Sr\\-S-\ S'Sr\\ S'Sr\\-S-\ S'Sr\\ S'Sr\\ S'S r\\ S!'Sr\S-\ S"'Sr\S-\ S#'Sr\\ \-S-\ S$'S%r!\ \ S&'U4S'jr"S(r#U=r$$))BltLocalDecoderConfig@r blt_local_decoderrrrTNrrrrrrrrrr rrrr r!r"r#r$r%r&i r'r(r) pad_token_id bos_token_id eos_token_idFr/c >UR=(d URUlURUR-UlUR=(d [ SUR-S- 5UlSUl[TU] "S0UD6 gr+ rrrhead_dimr'r.r/r0r1r2s r6r1#BltLocalDecoderConfig.__post_init__csu#'#;#;#Wt?W?W ((D,D,DD !%!7!7!X3q4CSCS?SVW?W;X#(  ''r8rRr'rr/)%r9r:r;r<r=r>r?rr.r@rrArrrrrrrrBr!r#r$rrCr&rDr'r)rMrNrOlistr/r1rErFrGs@r6rIrI@s%JMJ)-4$;- L#* %)d )K!!&*t*sL%"%GUS[4 %#(S(48O^d*T18J!s!#u##L#*##L#*#+/L#S /D(/ %%((r8rIc^\rSrSr%SrSrSr\\S'Sr \\S'Sr \S-\S 'S r \\S 'S r \ \S 'Sr\ \-S-\S'Sr\\S'Sr\\-S-\S'Sr\\S'Sr\\S'Sr\ \S'Sr\\S'U4SjrSrU=r$)BltGlobalTransformerConfigkblt_global_transformerrrrrrNrrrrr r!r#r$r%r&ir'r(r)Fr/c >UR=(d URUlURUR-UlUR=(d [ SUR-S- 5UlSUl[TU] "S0UD6 gr+rQr2s r6r1(BltGlobalTransformerConfig.__post_init__~su#'#;#;#Wt?W?W ((D,D,DD !%!7!7!X3q4CSCS?SVW?W;X#(  ''r8rT)r9r:r;r<r>r?rr.r@rrrrrBr!r#r$rrCr&rDr'r)r/rAr1rErFrGs@r6rWrWks*JMK!!&*t*sL%"%GUS[4 %#'S'48O^d*T18J!s!#u# %%((r8rWc^\rSrSr%SrSr\\S'Sr\\S'Sr \\S'S r \\S 'S r \S -\S 'S r \\S'Sr \\S'Sr\\-S -\S'Sr\\S'S r\\-S -\S'Sr\\S'Sr\\S'U4SjrSrU=r$)BltPatcherConfig blt_patcherrrirr rNri r#rrr r!rr'r$r(r)Fr/c ,>UR=(d URUlURUR-UlUR=(d [ SUR-S- 5UlSUlSUl[TU]$"S0UD6 g)Nr,rFr%r-) rrrrRr'r.r/r&r0r1r2s r6r1BltPatcherConfig.__post_init__s|#'#;#;#Wt?W?W ((D,D,DD !%!7!7!X3q4CSCS?SVW?W;X#(   ''r8)rRr&r'rr/)r9r:r;r<r>rr.r@rrrrr#rrBr!r'r$rrCr)r/rAr1rErFrGs@r6r_r_sJJKs!!&*t*#'S'L%"%GUS[4 %!s!48O^d*T18#u# %%((r8r_c^\rSrSr%SrSrS/rSr\\ \ \ S.r Sr \\S'S r\\S 'S r\S -\S 'Sr\S -\S'Sr\S -\S'Sr\S -\S'Sr\S -\S'S r\S -\S'Sr\S -\S'S r\\S -\S'Sr\S -\S'Sr\S -\S'S r\ \!-S -\S'S r"\ \!-S -\S'S r#\ \!-S -\S'S r$\ \!-S -\S 'S!r%\\S"'S r&\S -\S#'S r'\S -\S$'S r(\\\-S -\S%'S&r)\\S''S r*\+\ -S -\S('U4S)jr,S*r-U=r.$)+ BltConfiga patch_in_forward (`bool`, *optional*, defaults to `True`): Whether to perform patching during the forward pass. patch_size (`int`, *optional*, defaults to 4): Size of the patches used in the patching mechanism. patching_mode (`str`, *optional*, defaults to `"entropy"`): The mode used for patching, such as entropy-based patching. patching_threshold (`float`, *optional*, defaults to 1.34): Threshold value used for determining when to apply patches. patching_batch_size (`int`, *optional*, defaults to 1): Batch size used during the patching process. max_patch_length (`int`, *optional*): Maximum length of patches that can be generated. cross_attn_k (`int`, *optional*, defaults to 2): Number of cross-attention heads used in the model. encoder_hash_byte_group_size (`list`, *optional*): List of byte group sizes used in the encoder hash function. encoder_hash_byte_group_vocab (`int`, *optional*, defaults to 500002): Vocabulary size for the encoder hash byte groups. encoder_hash_byte_group_nb_functions (`int`, *optional*, defaults to 1): Number of hash functions used in the encoder byte grouping. patcher_config (`BltPatcherConfig`, *optional*): Configuration for the patcher component of the model. global_config (`BltGlobalTransformerConfig`, *optional*): Configuration for the global transformer component of the model. Example: ```python >>> from transformers import BltModel, BltConfig >>> # Initializing a Blt configuration >>> configuration = BltConfig() >>> # Initializing a model from the configuration >>> model = BltModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```bltpast_key_valuesr)patcher_configencoder_configdecoder_config global_configrrr[r#TNpatch_in_forward patch_sizeentropy patching_modeg]?patching_thresholdrpatching_batch_sizemax_patch_lengthrrencoder_hash_byte_group_sizei"encoder_hash_byte_group_vocab$encoder_hash_byte_group_nb_functionsrkrlrmrnFr/rMrNrOr(r)r$c >UR=(d /SQUlURc.[URS9Ul[R S5 O_[ UR[5(a@URRSUR5 [S0URD6UlURc.[URS9Ul [R S5 O_[ UR[5(a@URRSUR5 [S0URD6Ul URc.[URS9Ul [R S5 O_[ UR[5(a@URRSUR5 [S0URD6Ul URc.[URS9Ul [R S5 O_[ UR[5(a@URRSUR5 [S0URD6Ul URRUR -nX RR:waUOSURl[$TU]L"S0UD6 g) N)rrpr,)r)z8patcher_config is None, using default Blt patcher configr)z8encoder_config is None, using default Blt encoder configz8decoder_config is None, using default Blt decoder configz6global_config is None, using default Blt global configr-)rwrkr_r)loggerinfo isinstancerC setdefaultrlr rmrIrnrWrrencoder_cross_output_sizer0r1)r3r4rr5s r6r1BltConfig.__post_init__s8,0,M,M,cQc)    &"2TE[E["\D  KKR S ++T 2 2    * *+>@V@V W"2"IT5H5H"ID     &"7$J`J`"aD  KKR S ++T 2 2    * *+>@V@V W"7"N$:M:M"ND     &"7$J`J`"aD  KKR S ++T 2 2    * *+>@V@V W"7"N$:M:M"ND     %!;dNdNd!eD  KKP Q **D 1 1    ) )*=t?U?U V!;!Qd>P>P!QD %)$7$7$C$CdFWFW$W!)BFXFXFdFd)d %jn 4 ''r8)rmrlrwrnrk)/r9r:r;r<r=r>keys_to_ignore_at_inferencer?r_r rIrW sub_configsrr.r@r#rorArqrsrDrtrBrurvrrwrUrxryrkrCrrlrmrnr/rMrNrOr)r$rr1rErFrGs@r6rgrgs& PJ#4"5M*//3 KJ#'S'$(dTk(Jd  )M3:)'8 8&'t'#'cDj' L#* 59 $s)d"2906!3:678(#*859ND++d2959ND++d2959ND++d2948M4**T18 %%#L#*##L#*#+/L#S /D(/#u#48O^d*T18&(&(r8rg)rgr_r rIrWN)r=huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr get_loggerr9r~r rIrWr_rg__all__r-r8r6rs.31,   H %-.!(,!(/!(H-.&(,&(/&(R-.(!1(/(4-.('(/(4-.p( p(/p(f r8