Zj dSrSSKJr SSKJr SSKJr \"SS9\"SS \555rS /rg ) zBloom configuration)strict)PreTrainedConfig)auto_docstringzbigscience/bloom) checkpointcL^\rSrSr%SrSrS/rSSS.rSr\ \ S 'S r \ \ S 'S r \ \ S'S r \ \ S'Sr\\ S'Sr\\ S'Sr\\ S'Sr\ S-\ S'S r\ \\ -S-\ S'Sr\ S-\ S'Sr\\ S'Sr\\ -\ S'Sr\\ -\ S'Sr\ \ S'Sr\\ S'Sr\\ S 'U4S!jrS"rU=r$)# BloomConfiga apply_residual_connection_post_layernorm (`bool`, *optional*, defaults to `False`): If enabled, use the layer norm of the hidden states as the residual in the transformer blocks slow_but_exact (`bool`, *optional*, defaults to `False`): Experimental feature. Whether to use slow but exact implementation of the attention mechanism. While merging the TP rank tensors, due to slicing operations the results may be slightly different between the model trained on Megatron and our model. Please refer to [this issue](https://github.com/pytorch/pytorch/issues/76232). A solution to obtain more accurate results is to enable this feature. Enabling this will hurt the computational time of the inference. Will be probably resolved in the future once the main model has been fine-tuned with TP_rank=1. Example: ```python >>> from transformers import BloomConfig, BloomModel >>> # Initializing a Bloom configuration >>> configuration = BloomConfig() >>> # Initializing a model (with random weights) from the configuration >>> model = BloomModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```bloompast_key_valuesn_layern_head)num_hidden_layersnum_attention_headsi vocab_size@ hidden_sizegh㈵>layer_norm_epsilong{Gz?initializer_rangeT use_cacheN bos_token_id eos_token_id pad_token_idF(apply_residual_connection_post_layernormghidden_dropoutattention_dropoutpretraining_tpslow_but_exacttie_word_embeddingsc v>URSS5nUc UROUUl[TU] "S0UD6 g)Nn_embed)poprsuper __post_init__)selfkwargsr$ __class__s ~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/bloom/configuration_bloom.pyr(BloomConfig.__post_init__Ks7**Y-/64++G '')r) __name__ __module__ __qualname____firstlineno____doc__ model_typekeys_to_ignore_at_inference attribute_maprint__annotations__rr rrfloatrrboolrrlistrrrrr r!r"r(__static_attributes__ __classcell__)r+s@r,r r s 4J#4"5&'M JKGSFCO $$#u#It L#* +,L#S /D(,#L#*#5:,d:"%NECK%%(us{(NC ND $$((r.r N) r3huggingface_hub.dataclassesrconfiguration_utilsrutilsrr __all__r%r.r,rBsG.3#-.7("7(/7(t /r.