Zj1!SrSSKJr SSKJr SSKJr SSKJr SSK J r \"S S 9\"S S \555r \"S S 9\"S S\555r SS /r g)zMoshi model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstring) AutoConfigzkmhf/hf-moshiko) checkpointc^\rSrSr%SrSrS/rSr\\ S'Sr \\ S'S r \\ S 'S r \\ S 'S r \\ S'Sr\S-\ S'Sr\\ S'Sr\\ S'Sr\\ S'Sr\S-\ S'Sr\\ S'Sr\\ S'Sr\\ S'Sr\\-\ S'S r\\ S!'S"r\\ S#'Sr\\ S$'S%r\\ S&'Sr\S-\ S''Sr\S-\ S('Sr \\!\-S-\ S)'U4S*jr"S+r#S,r$U=r%$)-MoshiDepthConfiga input_size (`int`, *optional*, defaults to 4096): Dimensionality of the input hidden states. Used to connect the main decoder to the depth decoder. audio_vocab_size (`int`, *optional*, defaults to 2048): Vocabulary size of the audio part of model. Defines the number of different tokens that can be represented by the `audio_codes` passed when calling the Moshi models. ffn_dim (`int`, *optional*, defaults to 5632): Dimensionality of the "intermediate" (often named feed-forward) layer in the depth decoder block. Must be even. Example: ```python >>> from transformers import ( ... MoshiDepthConfig, ... MoshiDepthDecoder, ... ) >>> configuration = MoshiDepthConfig() >>> # Initializing a MoshiDepthDecoder (with random weights) from the kmhf/hf-moshiko style configuration >>> model = MoshiDepthDecoder(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` moshi_depthpast_key_values} vocab_sizei hidden_size input_sizenum_hidden_layersnum_attention_headsNnum_key_value_headsiaudio_vocab_size max_position_embeddingssilu hidden_acthead_dim{Gz?initializer_rangeT use_cachesliding_windowattention_dropoutiffn_dim:0yE> rms_norm_eps num_codebooksFtie_word_embeddings pad_token_id bos_token_id eos_token_idc >URb URO URUlUR=(d URUR-Ul[TU]"S0UD6 g)N)rrrrsuper __post_init__)selfkwargs __class__s ~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/moshi/configuration_moshi.pyr2MoshiDepthConfig.__post_init__Ns[(,(@(@(LD $ $RVRjRj   U)9)9T=U=U)U  ''c\URS-S:Xa[SURS35eg)OPart of `@strict`-powered validation. Validates the architecture of the config.r `ffn_dim=` must be even.N)r' ValueErrorr3s r6validate_architecture&MoshiDepthConfig.validate_architectureUs0 <>> from transformers import ( ... MoshiConfig, ... MoshiForConditionalGeneration, ... ) >>> configuration = MoshiConfig() >>> # Initializing a MoshiForConditionalGeneration (with random weights) from the kmhf/hf-moshiko style configuration >>> model = MoshiForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # Saving the model, including its configuration >>> model.save_pretrained("kmhf/hf-moshiko") >>> # loading model and config from pretrained folder >>> moshi_config = MoshiConfig.from_pretrained("kmhf/hf-moshiko") >>> model = MoshiForConditionalGeneration.from_pretrained("kmhf/hf-moshiko", config=moshi_config) ```moshir)audio_encoder_configdepth_decoder_configrrrr rrNrri rrope_parametersrrrr r!Tr"r$r%r&iXr'r(r)r#r*Fr+r,r-r.rVrWc >URb URO URUlUR=(d URUR-Ul[ UR [ 5(aCUR RSS5n[R"U40UR D6UlO(UR c[R"S5UlURcUR RO URUl [ UR[ 5(acURRURURURURS.5 [!S0URD6Ul OURc[!5Ul ["TU]H"S0UD6 g)NrGmimi)rrrr*r0)rrrr isinstancerVdictpopr for_modelr codebook_sizerWupdaterr*r r1r2)r3r4audio_encoder_model_typer5s r6r2MoshiConfig.__post_init__ss(,(@(@(LD $ $RVRjRj   U)9)9T=U=U)U d// 6 6'+'@'@'D'D\SY'Z $(2(<(<=U(sY]YrYr(sD %  & & .(2(<(r*rVr?s r6r@!MoshiConfig.validate_architectures <rws .31#0,->H'>H.>HB,-| "| .| ~ , -r8