Zj!lSSKJr SSKJr SSKJr SSKJr \"SS9\"SS \555rS /r g ) )strict)PreTrainedConfig)RopeParameters)auto_docstringzUsefulSensors/moonshine-tiny) checkpointc^\rSrSr%SrSrS/rSSSSS .rS r\ \ S 'S r \ \ S 'Sr \ \ S'Sr \ \ S'Sr\ \ S'Sr\ \ S'Sr\ \ S'Sr\ S-\ S'Sr\ S-\ S'Sr\ S-\ S'Sr\\ S'Sr\\ S'Sr\ \ S'Sr\\ S'Sr\ \ S'S r\\ S!'Sr\\-S-\ S"'S r \\ S#'S$r!\\ S%'S&r"\\ -\ S''Sr#\ S-\ S('S)r$\ \%\ -S-\ S*'Sr&\ S-\ S+'S r'\\ S,'U4S-jr(S.r)U=r*$)/MoonshineConfiga encoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `encoder_num_key_value_heads=encoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `encoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `num_attention_heads`. decoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `decoder_num_key_value_heads=decoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `decoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `decoder_num_attention_heads`. pad_head_dim_to_multiple_of (`int`, *optional*): Pad head dimension in encoder and decoder to the next multiple of this value. Necessary for using certain optimized attention implementations. encoder_hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder. decoder_hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the decoder. Example: ```python >>> from transformers import MoonshineModel, MoonshineConfig >>> # Initializing a Moonshine style configuration >>> configuration = MoonshineConfig().from_pretrained("UsefulSensors/moonshine-tiny") >>> # Initializing a model from the configuration >>> model = MoonshineModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` moonshinepast_key_valuesdecoder_num_key_value_headsdecoder_num_attention_headsdecoder_num_hidden_layersdecoder_hidden_act)num_key_value_headsnum_attention_headsnum_hidden_layers hidden_acti vocab_sizei  hidden_sizeiintermediate_sizeencoder_num_hidden_layersencoder_num_attention_headsNencoder_num_key_value_headspad_head_dim_to_multiple_ofgeluencoder_hidden_actsiluimax_position_embeddingsg{Gz?initializer_rangedecoder_start_token_idT use_cacherope_parametersis_encoder_decoderFattention_biasgattention_dropout bos_token_id eos_token_id pad_token_idtie_word_embeddingsc >URcURUlURcURUlUR SS5 [ TU]"S0UD6 g)Npartial_rotary_factorg?)rrrr setdefaultsuper __post_init__)selfkwargs __class__s چ/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/moonshine/configuration_moonshine.pyr5MoonshineConfig.__post_init__isX  + + 3/3/O/OD ,  + + 3/3/O/OD ,137 '')rr)+__name__ __module__ __qualname____firstlineno____doc__ model_typekeys_to_ignore_at_inference attribute_maprint__annotations__rrrrrrrrrr strrr"r#floatr%r&boolr'rdictr(r)r*r+r-listr.r/r5__static_attributes__ __classcell__)r8s@r9r r sg& PJ#4"5<<8* MJK!s!%&s&%&s&'(('((.2t2.2t2.2t2$$$$#&S&#u#"#C#It48O^d*T18## ND %(us{( L#* +,L#S /D(,#L#*# $$((r;r N) huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr __all__r2r;r9rRsK*/31#9:S(&S(;S(l  r;