ZjXhSSKJr SSKJr SSKJr SSKJr SSKJ r J r J r J r \ "5(aSSK JrJr \ R "\5r\ "SS 9\"S S \555r\ "SS 9\"S S \555r\ "SS 9\"SS\555r\ "SS 9\"SS\555r/SQrg))Sequence)Any)strict)PreTrainedConfig)auto_docstringis_timm_availableloggingrequires_backends) ImageNetInfoinfer_imagenet_subsetzgoogle/gemma-3n-E4B) checkpointc ^\rSrSr%SrSrS/rSSSSSSSSSSS. rS /S /4S S /S /4S /S /4S .rSr \ \ S'Sr \ \ S'Sr \ \\ -\ S'Sr\ \ S'Sr\ \ S'Sr\ \ S'Sr\ \ S'Sr\\ S'Sr\ \ S'S r\\ S!'S"r\\ S#'S$r\\ S%'S&r\ S'-\ S('S)r\ \\ -S'-\ S*'Sr\ S'-\ S+'S$r\\ S,'S'r\ S'-\ S-'S.r!\\ S/'S0r"\ \-S'-\ S1'S2r#\ \ S3'S'r$\\S'-\ S4'S5r%\\ S6'S7S8S9.r&S:r'\ \ S;'Sr(\ \ S<'S&r)\ \ S='S>r*\\ S?'S$r+\\ S@'SAr,\ \ SB'SCr-\ \ SD'SEr.\ \ SF'S'r/\\\-S'-\ SG'U4SHjr0SIr1SJr2SKr3U=r4$)LGemma3nTextConfig$a vocab_size_per_layer_input (`int`, *optional*, defaults to 262144): Vocabulary size of the per-layer text embeddings that augment the standard embeddings. hidden_size_per_layer_input (`int`, *optional*, defaults to 256): Dimension of the hidden representations for per-layer embeddings. altup_active_idx (`int`, *optional*, defaults to 0): The index of the prediction from which AltUp will compute additional predictions or correct the active prediction. altup_coef_clip (`float`, *optional*, defaults to 120.0): The maximum amplitude of an AltUp prediction or correction coefficient weight. altup_correct_scale (`bool`, *optional*, defaults to `True`): If True, apply the `AltUp.correct_output_scale` to the corrected prediction at `altup_active_idx`. altup_num_inputs (`int`, *optional*, defaults to 4): The number of predictions that AltUp should make given the input sequence. num_kv_shared_layers (`int`, *optional*, defaults to 15): The number of layers that share KV cache values. During the forward pass, the last `num_kv_shared_layers` layers in the model "share" the KV values in that each local and global layer in this range uses the KV cache values computed for the last local or global layer, respectively, before entering this range. The value should be a multiple of the attention pattern size (see `layer_types` parameter). laurel_rank (`int`, *optional*, defaults to 64): The intermediate size for the linear projections in the Learned Augmented Residual Layer. activation_sparsity_pattern (`Sequence[float]`, *optional*): The sparsity factor used to extract the top-k activations for a given layer. The provided Sequence must explicitly provide a sparsity value for each layer in the model. By default, the first 10 layers are sparse with a sparsity factor of 0.95 and the rest are dense. ```python >>> from transformers import Gemma3nTextModel, Gemma3nTextConfig >>> # Initializing a Gemma3nText gemma3n_text-E4B style configuration >>> configuration = Gemma3nTextConfig() >>> # Initializing a model from the gemma3n_text-E4B style configuration >>> model = Gemma3nTextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` gemma3n_textpast_key_valuescolwisereplicated_with_grad_allreducerowwise) zlayers.*.self_attn.q_projzlayers.*.self_attn.k_projzlayers.*.self_attn.v_projzlayers.*.self_attn.q_normzlayers.*.self_attn.k_normzlayers.*.self_attn.v_normzlayers.*.self_attn.o_projzlayers.*.mlp.gate_projzlayers.*.mlp.up_projzlayers.*.mlp.down_proj input_ids inputs_embeds hidden_statesattention_mask) embed_tokenslayersnormi vocab_size hidden_sizei@intermediate_size#num_hidden_layersnum_attention_headsnum_key_value_headshead_dimgelu_pytorch_tanhhidden_activationimax_position_embeddings{Gz?initializer_rangeư> rms_norm_epsT use_cacherN pad_token_id eos_token_id bos_token_idtie_word_embeddingsrope_parametersFattention_biasattention_dropoutisliding_window layer_typesg>@final_logit_softcappingg.Ag@)globallocalvocab_size_per_layer_inputhidden_size_per_layer_inputaltup_active_idxg^@altup_coef_clipaltup_correct_scalealtup_num_inputsnum_kv_shared_layers@ laurel_rankactivation_sparsity_patternc >[UR[5(a@[UR5=o R:wa[ SURSUS35e[UR[5(dUR/UR-UlUR c8[UR5Vs/sHo3S-S-S:XaSOSPM snUlURc3URS :aS OSnS /U-S /URU- --Ul[UR5=oPR:wa[ S URSUS35e[TU]("S 0UD6 gs snf)Nzjintermediate_size must have an explicit intermediate size for every layer or one for all layers. Expected z values but got .r3rfull_attentionsliding_attention gffffff?r9zeactivation_sparsity_pattern must have an explicit activation sparsity value for every layer.Expected ) isinstancer!rlenr# ValueErrorr<rangerLsuper __post_init__)selfkwargs intsize_leninum_sparse_layerslen_asp __class__s ڂ/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/gemma3n/configuration_gemma3n.pyrYGemma3nTextConfig.__post_init__s t--x 8 8 #D$:$: ;;@V@VV 2233CK=PQS D22H==&*&<&<%=@V@V%VD "    #W\]a]s]sWt WtRSUaK1$4 :MMWt D   + + 3&*&<&