ZjP"SrSSKJr SSKJr SSKJrJr \"5(aSSKJ r J r J r J r J r JrJrJr SrO1SSKJr \S r \S r \S r \S r \S r \SrS\S\S\4SjrSr\"SS9\"SS\555rS/rg)zxLSTM configuration.)strict)PreTrainedConfig)auto_docstringis_xlstm_available)BackendModeTypeChunkwiseKernelType DtypeTypeSequenceKernelTypeStepKernelTypeWeightModeTyperound_up_to_next_multiple_ofxLSTMLargeConfigT)Literal)traintrain_with_padding inference)chunkwise--native_autogradzparallel--native_autograd)float32bfloat16float16native_sequence__nativenative)singlefusedx multiple_ofreturnc.[X-S- U-U-5$)z0Rounds up x to the next multiple of multiple_of.)int)rrs ~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/xlstm/configuration_xlstm.pyrr1sQ_q([8KGHHFzNX-AI/xLSTM-7b) checkpointcb^\rSrSr%SrSrSr\\S'Sr \\S'Sr \S-\S 'S r \\S 'Sr \S-\S 'S r \\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S 'S!r\\S"'S#r\\S$'Sr \\S%'S&r!\"\S''Sr#\\S('S)r$\"\S*'S+r%\\S,'S#r&\\S-'S.r'\\S/'S0r(\\S1'S2r)\*\S3'Sr+\\S4'S5r,\S-\S6'S7r-\S-\S8'S9r.\\/\-S-\S:'S;r0\\S<'U4S=jr1\2S>5r3\2S?5r4\2S@5r5\2SA5r6SBr7SCr8U=r9$)D xLSTMConfig8a~ num_blocks (int, optional, *optional*, defaults to 32): Number of blocks of the xLSTM model, use num_hidden_layers if None. num_heads (int, optional, *optional*, defaults to 8): Number of heads for the xLSTM Layer/Cell. use_bias (bool, optional, *optional*, defaults to `False`): Whether to use biases in the xLSTM model. norm_reduction_force_float32 (bool, optional, *optional*, defaults to `True`): Whether to force the float32 norm reduction op to be done in fp32 precision. add_out_norm (bool, optional, *optional*, defaults to `True`): Whether to add an output norm after the blocks before the LMHead. qk_dim_factor (float, optional, *optional*, defaults to 0.5): Scale factor for the query and key dimension. v_dim_factor (float, optional, *optional*, defaults to 1.0): Scale factor for the value dimension. chunkwise_kernel (ChunkwiseKernelType, optional, *optional*, defaults to `"chunkwise--native_autograd"`): Kernel type for chunkwise processing mode. sequence_kernel (SequenceKernelType, optional, *optional*, defaults to `"native_sequence__native"`): Kernel type for sequence processing mode. step_kernel (StepKernelType, optional, *optional*, defaults to `"native"`): Kernel type for step processing mode. mode (BackendModeType, optional, *optional*, defaults to `"inference"`): Operation mode (inference is needed for generation). chunk_size (int, optional, *optional*, defaults to 64): Internal chunk size. return_last_states (bool, optional, *optional*, defaults to `True`): If to return the last states / cache internally. Needed as True for generation. autocast_kernel_dtype (DtypeType, optional, *optional*, defaults to `"bfloat16"`): Kernel dtype for the states. inference_state_dtype (DtypeType, optional, *optional*, defaults to `"float32"`): Kernel dtype for states in inference. ffn_proj_factor (float, optional, *optional*, defaults to 2.667): Size factor of the post-up projection gated Feed Forward network. ffn_round_up_to_multiple_of (int, optional, *optional*, defaults to 64): Size factor round value of the post-up projection gated Feed Forward network. gate_soft_cap (float, optional, *optional*, defaults to 15.0): Gate soft cap scale. output_logit_soft_cap (float, optional, *optional*, defaults to 30.0): Output logit soft cap scale. weight_mode (`Literal`, *optional*, defaults to `"single"`): Whether parallel linear layers are separated or fused (single). max_inference_chunksize (int, optional, *optional*, defaults to 16384): Limit the chunk size for inference to save memory. Example: ```python >>> from transformers import xLSTMConfig, xLSTMModel >>> # Initializing a xLSTM configuration >>> configuration = xLSTMConfig() >>> # Initializing a model (with random weights) from the configuration >>> model = xLSTMModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```xlstmi vocab_sizei hidden_sizeN embedding_dim num_hidden_layers num_blocks num_headsFuse_biasTnorm_reduction_force_float32tie_word_embeddings add_out_normgư>norm_epsg? qk_dim_factorg? v_dim_factorrchunkwise_kernelrsequence_kernelr step_kernelrmode@ chunk_sizereturn_last_statesrautocast_kernel_dtypeepsrinference_state_dtypegtV@ffn_proj_factorffn_round_up_to_multiple_ofg.@ gate_soft_capg>@output_logit_soft_capr weight_mode use_cacher pad_token_idr bos_token_id eos_token_idi@max_inference_chunksizec v>URb URO URUlURb URO URUlURb URO URUlURb URO URUl[TU]"S0UD6 g)N)r*r+r-r.super __post_init__)selfkwargs __class__s r"rPxLSTMConfig.__post_init__s/3/?/?/K4++QUQcQc373E3E3QT//W[WgWg;?;Q;Q;]!7!7cgcrcr-1__-H$//dNdNd ''r#cD[URUR-SS9$Nr<)r)rr*r6rQs r"qk_dimxLSTMConfig.qk_dims&+   t11 1  r#cD[URUR-SS9$rV)rr*r7rWs r"v_dimxLSTMConfig.v_dims&+   t00 0  r#c4URUR-$N)rXr0rWs r" qk_head_dimxLSTMConfig.qk_head_dims{{dnn,,r#c4URUR-$r^)r[r0rWs r" v_head_dimxLSTMConfig.v_head_dimszzT^^++r#c[(GaA[S0SUR_SUR_SUR_SUR _SUR _SUR_SUR_SUR_S UR_S UR_S UR_S UR_S UR_SUR_SUR _SUR"_SUR$_SUR&_SUR(_SUR*_SUR,_SUR._SUR0_SUR2_6$U$)Nr)r+r.r0r1r4r5r2r6r7r8r9r:r;r=r>r?r@rArBrCrDrErFrN)external_xlstmrr)r*r-r0r1r4r5r2r6r7r8r9r:r;r=r>r?r@rArBrCrDrErFrWs r"to_xlstm_block_config!xLSTMConfig.to_xlstm_block_configs >#??".. 11..    ".. .2-N-N#00".."&!6!6!% 4 4!,, YY!" ??#$$(#:#:%&'+&@&@'(HH)*'+&@&@+.!% 4 4/0-1,L,L14#0056'+&@&@78!,,9 >Kr#)r+r*r.r-):__name__ __module__ __qualname____firstlineno____doc__ model_typer)r!__annotations__r*r+r-r.r0r1boolr2r3r4r5floatr6r7r8r r9r r:r r;rr=r>r?r r@rArBrCrDrErFr rGrHrIrKlistrLrPpropertyrXr[r_rbrf__static_attributes__ __classcell__)rSs@r"r&r&8s9 vJJK $M3:$s!Jd !IsHd)- $- %%L$HeM5L%,H)H*CO'C"*K*'D/'J##'191C'090"OU"'))M5#'5'"*K*It L#*  L#* +,L#S /D(,#(S((    --,,!!r#r&N)rlhuggingface_hub.dataclassesrconfiguration_utilsrutilsrrxlstm.xlstm_large.modelrr r r r r rrretypingrr!r&__all__rNr#r"r{s.37   NHIO! %89I !:;X&N./NII#I#IN+,]"]-]@ /r#