Zj{IbSrSSKJr SSKJr SSKJr SSKJrJ r \ R"\ 5r \"SS9\"S S \555r \"SS9\"S S \555r\"SS9\"S S\555r\"SS9\"SS\555r\"SS9\"SS\555r/SQrg)zFLAVA model configurations)Any)strict)PreTrainedConfig)auto_docstringloggingzfacebook/flava-full) checkpointc8\rSrSr%SrSrSrSr\\ S'Sr \\ S'Sr \\ S 'S r \\ S 'S r \\ S 'Sr\\-\ S'Sr\\-\ S'Sr\\ S'Sr\\ S'Sr\\\-\\\4-\ S'Sr\\\-\\\4-\ S'Sr\\ S'Sr\\ S'Sr\\ S'Sr\\ S'S rg!)"FlavaImageConfiga$ mask_token (`bool`, *optional*, defaults to `True`): Whether to use a mask token or not. Used in MIM (Masked Image Modeling) loss for FLAVA. Example: ```python >>> from transformers import FlavaImageConfig, FlavaImageModel >>> # Initializing a FlavaImageModel with style configuration >>> configuration = FlavaImageConfig() >>> # Initializing a FlavaImageModel model (with random weights) from the style configuration >>> model = FlavaImageModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```flava_image_model image_config hidden_size num_hidden_layersnum_attention_heads intermediate_sizegelu hidden_acthidden_dropout_probattention_probs_dropout_prob{Gz?initializer_range-q=layer_norm_eps image_size patch_sizer num_channelsTqkv_bias mask_token vocab_sizeN)__name__ __module__ __qualname____firstlineno____doc__ model_typebase_config_keyrint__annotations__rrrrstrrfloatrrrr listtupler"r#r$boolr%r'__static_attributes__r(~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/flava/configuration_flava.pyr r s &%J$OKs!!!s!J'**03 %#+3#u#!NE!47Jd3i%S/1746Jd3i%S/16L#HdJJr8r c\rSrSr%SrSrSrSr\\ S'Sr \\ S'S r \\ S 'S r \\ S 'S r \\ S'S r\\ S'Sr\\ S'Sr\\ S'Sr\\-\ S'Sr\\-\ S'Sr\\ S'Sr\\ S'Sr\S-\ S'Sr\\ S'S rg)!FlavaTextConfigEa Example: ```python >>> from transformers import FlavaTextConfig, FlavaTextModel >>> # Initializing a FlavaTextModel with style configuration >>> configuration = FlavaTextConfig() >>> # Initializing a FlavaTextModel model (with random weights) from the style configuration >>> model = FlavaTextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```flava_text_model text_configi:wr'type_vocab_sizeimax_position_embeddingsrrrrrrrrrrrrrrrrrN pad_token_idTr$r()r)r*r+r,r-r.r/r'r0r1r@rArrrrrr2rr3rrrrBr$r6r7r(r8r9r;r;Es $J#OJOS#&S&Ks!!!s!J'**03 %#+3#u#!NE! L#* Hdr8r;c\rSrSr%SrSrSrSr\\ S'Sr \\ S'S r \\ S 'S r \\ S 'S r \\ S'Sr\\-\ S'Sr\\-\ S'Sr\\ S'Sr\\ S'Sr\\ S'Sr\\ S'Srg)FlavaMultimodalConfigkaK use_cls_token (`bool`, *optional*, defaults to `True`): Whether to use an extra CLS token for multimodal settings. Usually needed by the FLAVA model. Example: ```python >>> from transformers import FlavaMultimodalConfig, FlavaMultimodalModel >>> # Initializing a FlavaMultimodalModel with style configuration >>> configuration = FlavaMultimodalConfig() >>> # Initializing a FlavaMultimodalModel model (with random weights) from the style configuration >>> model = FlavaMultimodalModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```flava_multimodal_modelmultimodal_configrrrrrrrrrrrrrrrrTr$ use_cls_tokenr(N)r)r*r+r,r-r.r/rr0r1rrrrr2rr3rrrr$r6rIr7r(r8r9rDrDks &*J)OKs!!!s!J'**03 %#+3#u#!NE!HdM4r8rDc|\rSrSr%SrSr\\S'Sr\\S'Sr \\S'S r \\S 'S r \\S 'S r \ \S'Sr\\S'Srg)FlavaImageCodebookConfiga+ num_groups (`int`, *optional*, defaults to 4): Number of groups to be created. This parameter as of now doesn't affect the model and is used for some internal calculation and estimations. num_blocks_per_group (`int`, *optional*, defaults to 2): Number of conv-based blocks per group. freeze (`bool`, defaults to `True`): Whether to freeze the weights of the model. Example: ```python >>> from transformers import FlavaImageCodebookConfig, FlavaImageCodebook >>> # Initializing a FlavaImageCodebook with style configuration >>> configuration = FlavaImageCodebookConfig() >>> # Initializing a FlavaImageCodebook model (with random weights) from the style configuration >>> model = FlavaImageCodebook(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```  num_groupsrinput_channelsr?num_blocks_per_grouprr&r'Tfreezerrr(N)r)r*r+r,r-rNr0r1rOrPrr'rRr6rr3r7r(r8r9rKrKsQ0JNC !#!KJFD#u#r8rKc^\rSrSr%SrSr\\\\ S.r Sr \ \ \4\-S-\S'Sr\ \ \4\-S-\S'Sr\ \ \4\-S-\S'Sr\ \ \4\-S-\S 'S r\\S 'S r\\S 'S r\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr\\S'Sr \\S'Sr!\\S'Sr"\\S'Sr#\\S'Sr$\\S'Sr%\\S'Sr&\\S 'Sr'\\S!'Sr(\\S"'U4S#jr)S$r*U=r+$)% FlavaConfigaP image_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`FlavaImageConfig`]. multimodal_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`FlavaMultimodalConfig`]. image_codebook_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`FlavaCodebookConfig`]. init_codebook (`bool`, *optional*, defaults to `True`): Whether to initialize the codebook logit_scale_init_value (`float`, *optional*, defaults to 2.6592): The initial value of the *logit_scale* parameter. Default is used as per the original FLAVA/CLIP implementation. ce_ignore_index (`int`, *optional*, defaults to -100): Cross entropy index to ignore. mim_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to MIM (Masked Image Modeling) unimodal loss mlm_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to MLM (Masked Language Modeling) unimodal loss global_contrastive_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to global contrastive cross-alignment loss. itm_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to image-text matching multimodal loss. mmm_image_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to MMM loss's image part. mmm_text_weight (`float`, *optional*, defaults to 1.0): Weight to be assigned to MMM loss's text part. global_backprop_contrastive (`bool`, *optional*, defaults to `True`): Whether to use global backpropgation through all workers in contrastive loss. skip_unmasked_multimodal_encoder (`bool`, *optional*, defaults to `True`): Whether to skip running unmasked multimodal encoder whose outputs are not used by FLAVA losses. return_loss (`bool`, *optional*, defaults to `True`): Whether to return loss or not Example: ```python >>> from transformers import FlavaConfig, FlavaModel, FlavaForPreTraining >>> # Initializing a FlavaConfig with style configuration >>> configuration = FlavaConfig() >>> # Initializing a FlavaModel and FlavaForPreTraining model (with random weights) from the style configuration >>> model = FlavaModel(configuration) >>> model_pre = FlavaForPreTraining(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> configuration_pre = model_pre.config ``` flava)r>rrGimage_codebook_configNrr>rGrWrrrrprojection_dimT init_codebookg/L F@logit_scale_init_valuerrice_ignore_indexg? mim_weight mlm_weightglobal_contrastive_weight itm_weightmmm_image_weightmmm_text_weightglobal_backprop_contrastive skip_unmasked_multimodal_encoder return_losstie_word_embeddingsinitializer_factorc X >URc0n[RS5 OF[UR[5(aURR 5nO URnUR c0n[RS5 OF[UR [5(aUR R 5nO UR nURc0n[RS5 OF[UR[5(aURR 5nO URnURc0n[RS5 OF[UR[5(aURR 5nO URnURSS5nURSS5nURSS5nURSS5n Ub[ S0UD6R 5n U R5HHupX;dM XU :wdMU S :wdMX;a S U S U S 3n OS U S3n [RU 5 MJ URU 5 Ub[S0UD6R 5nSU;a5USR5V V s0sHup[U 5U _M sn n US'UR5HHupX;dM XU :wdMU S :wdMX;a S U SU S 3n OSU S3n [RU 5 MJ URU5 Ub[S0UD6R 5nUR5HHupX;dM XU :wdMU S :wdMX;a S U SU S 3n OSU S3n [RU 5 MJ URU5 U b[S0U D6R 5nUR5HHupX;dM XU :wdMU S :wdMX;a S U SU S 3n OSU S3n [RU 5 MJ URU5 [ S0UD6Ul[S0UD6Ul[S0UD6Ul[S0UD6Ul [ TU]D"S0UD6 gs sn n f)NzP`text_config` is `None`. Initializing the `FlavaTextConfig` with default values.zR`image_config` is `None`. initializing the `FlavaImageConfig` with default values.z\`multimodal_config` is `None`. Initializing the `FlavaMultimodalConfig` with default values.zc`image_codebook_config` is `None`. initializing the `FlavaImageCodebookConfig` with default values.text_config_dictimage_config_dictmultimodal_config_dictimage_codebook_config_dicttransformers_version`zp` is found in both `text_config_dict` and `text_config` but with different values. The value `text_config_dict["z"]` will be used instead.zk`text_config_dict` is provided which will be used to initialize `FlavaTextConfig`. The value `text_config["z"]` will be overridden.id2labelzs` is found in both `image_config_dict` and `image_config` but with different values. The value `image_config_dict["zn`image_config_dict` is provided which will be used to initialize `FlavaImageConfig`. The value `image_config["z` is found in both `multimodal_config_dict` and `multimodal_config` but with different values. The value `multimodal_config_dict["z}`multimodal_config_dict` is provided which will be used to initialize `FlavaMultimodalConfig`. The value `multimodal_config["z` is found in both `image_codebook_config_dict` and `image_codebook_config` but with different values. The value `image_codebook_config_dict["z`image_codebook_config_dict` is provided which will be used to initialize `FlavaImageCodebookConfig`. The value `image_codebook_config["r()r>loggerinfo isinstancer;to_dictrr rGrDrWrKpopitemsupdater2super __post_init__)selfkwargsr>rrGrWrhrirjrk_text_config_dictkeyvaluemessage_image_config_dict_multimodal_config_dict_image_codebook_config_dict __class__s r9rwFlavaConfig.__post_init__ s    #K KKj k ((/ : :**224K**K    $L KKl m ))+; < <,,446L,,L  ! ! ) "  KKv w ..0E F F $ 6 6 > > @  $ 6 6   % % -$& ! KKu 224L M M$($>$>$F$F$H !$($>$> !"::&8$?"JJ':DA!',Dd!K%+ZZ0Ld%S"  ' / C2B C K K M 0557 %%s3C*COeHe.u%<T&U&]&]&_ #6;;= +39O0OTW[qTq4u%TTWSXXqs VVYUZZqs KK(>"  $ $%< = % 1*B*`E_*`*h*h*j ':??A 0s!;;558u%]]`\ab'' ]]`\aaxz KK('B, " ( ()D E+9[9,<|<!6!K9J!K%=%V@U%V" ''k2s7R&)rWrrGr>),r)r*r+r,r-r.r;r rDrK sub_configsrdictr2rrr1r>rGrWrr0rr3rXrYr6rZrr[r\r]r^r_r`rarbrcrdrerfrwr7 __classcell__)rs@r9rTrTsd1fJ&(2!9 K>BL$sCx.#33d:A<@Kc3h"22T9@BFtCH~(884?FFJ4S>,<rs !.3,   H %01%'%2%P01!&!2!H01!,!2!H01$/$2$D01q("q(2q(h vr8