ZjlSrSSKrSSKJr SSKJr SSKJr \"SS9\"S S \555rS /r g) zMAMBA configurationN)strict)PreTrainedConfig)auto_docstringzstate-spaces/mamba-2.8b) checkpointc^\rSrSr%SrSrSr\\S'Sr \\S'Sr \\S 'S r \\S 'S r \ \S 'Sr\S-\S'Sr\S-\S'Sr\\\-S-\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\ \S'Sr\\S'S r\\-\S!'S"r\ \S#'S$r\ \S%'Sr\ \S&'S'r\\S('S)r \ \S*'Sr!\\S+'Sr"\\S,'Sr#\\S-'Sr$\\S.'Sr%\\S/'U4S0jr&\'S15r(S2r)U=r*$)3 MambaConfiga layer_norm_epsilon (`float`, *optional*, defaults to 1e-05): The epsilon to use in the layer normalization layers. expand (`int`, *optional*, defaults to 2): Expanding factor used to determine the intermediate size. use_bias (`bool`, *optional*, defaults to `False`): Whether or not to use bias in ["in_proj", "out_proj"] of the mixer block use_conv_bias (`bool`, *optional*, defaults to `True`): Whether or not to use bias in the convolution layer of the mixer block. residual_in_fp32 (`bool`, *optional*, defaults to `True`): Whether or not residuals should be in `float32`. If set to `False` residuals will keep the same `dtype` as the rest of the model rescale_prenorm_residual (`bool`, *optional*, defaults to `False`): Whether or not to rescale `out_proj` weights when initializing. use_mambapy (`bool`, *optional*, defaults to `False`): Determines the fallback strategy during training if the CUDA-based official implementation of Mamba is not available. If `True`, the mamba.py implementation is used. If `False`, the naive and slower implementation is used. Consider switching to the naive version if memory is limited. use_associative_scan (`bool`, *optional*, defaults to `True`): Whether to use PyTorch's `torch._higher_order_ops.associative_scan` for the parallel scan instead of the naive sequential implementation. The associative scan is only active during `torch.compile` tracing and requires torch >= 2.9.0. Both paths are tested to produce numerically identical results (see `test_associative_scan_matches_sequential`). Set to `False` to fall back to the sequential loop. Example: ```python >>> from transformers import MambaConfig, MambaModel >>> # Initializing a Mamba configuration >>> configuration = MambaConfig() >>> # Initializing a model (with random weights) from the configuration >>> model = MambaModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```mambaih vocab_sizei hidden_size state_size num_hidden_layersgh㈵>layer_norm_epsilonrN pad_token_id bos_token_id eos_token_idexpand conv_kernelFuse_biasT use_conv_biassilu hidden_actg?initializer_rangeresidual_in_fp32autotime_step_rankg?time_step_scalegMbP? time_step_min time_step_maxrandomtime_step_init_schemeg-C6?time_step_floorrescale_prenorm_residual use_cache use_mambapyuse_associative_scantie_word_embeddingsc >[URUR-5UlURS:Xa#[ R "URS- 5O URUl[TU] "S0UD6 g)Nr r) intrr intermediate_sizer!mathceilsuper __post_init__)selfkwargs __class__s ~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/mamba/configuration_mamba.pyr4MambaConfig.__post_init__^sf!$T[[43C3C%C!D040C0Cv0MDIId&&+ ,SWSfSf  ''c"S/UR-$)Nr )r)r5s r8 layer_typesMambaConfig.layer_typesesy41111r:)r0r!)+__name__ __module__ __qualname____firstlineno____doc__ model_typer r/__annotations__r rrrfloatrrrlistrrrboolrrstrrrr!r"r#r$r&r'r(r)r*r+r,r4propertyr<__static_attributes__ __classcell__)r7s@r8r r sM$ LJJKJs $$ L#*  L#* +,L#S /D(,FCOKHdM4J"u"!d! &NC#I& OU  M5 M5!)3)!OU!%*d*ItK!%$% $$(22r:r ) rBr1huggingface_hub.dataclassesrconfiguration_utilsrutilsrr __all__r.r:r8rPsM .3#45M2"M26M2` /r: