Z j> S r SSKJr SSKJr SSKJr \" SS9\ " S S \5 5 5 r\" SS9\ " S
S \5 5 5 r\" SS9\ " S S
\5 5 5 r / SQr
g)zIdefics model configuration )strict )PreTrainedConfig)auto_docstringzHuggingFaceM4/idefics-9b)
checkpointc \ rS rSr% SrSS0rSr\\S' Sr \\
\ - \ \\4 - \S' Sr \\S ' S
r
\\
\ - \ \\4 - \S ' S r\\S
' Sr\\S' Sr\\S' Sr\\S' Sr\\S' Sr\\- \S' Sr\\S' Sr\\S' S rg ) IdeficsVisionConfig idefics_vision hidden_size embed_dimi
image_sizei intermediate_size
patch_size num_hidden_layers num_attention_headsr num_channelsgelu
hidden_actgh㈵>layer_norm_eps attention_dropout{Gz?initializer_rangeg ?initializer_factor N)__name__
__module__ __qualname____firstlineno__
model_type
attribute_mapr
int__annotations__r listtupler r r r r r strr floatr r r __static_attributes__r ڂ/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/idefics/configuration_idefics.pyr r s "J"K0MIs47Jd3i%S/17 !s!46Jd3i%S/16 s !!L# J NE %(us{( #u# ##r. r c r \ rS rSr% SrSrSr\\S' Sr \
\S' Sr \
\S ' S
r \
\S ' S r
\
\S
' Sr\\S' Srg)IdeficsPerceiverConfig/ a
use_resampler (`bool`, *optional*, defaults to `False`):
Whether or not to use the resampler
resampler_n_latents (`int`, *optional*, defaults to 64):
Number of latent embeddings to resample ("compress") the input sequence to (usually < 128).
resampler_depth (`int`, *optional*, defaults to 6):
Depth of the Perceiver Resampler (Transformer w/ cross attention). Should be shallow (< 3).
resampler_n_heads (`int`, *optional*, defaults to 16):
Number of heads in each Transformer block (for multi-headed self-attention).
resampler_head_dim (`int`, *optional*, defaults to 96):
Dimensionality of each head projection in the Transformer block.
qk_layer_norms_perceiver (`bool`, *optional*, defaults to `False`):
Whether or not to use qk layer norms in perceiver
idefics_percieverF
use_resampler@ resampler_n_latents resampler_depthr resampler_n_heads` resampler_head_dimqk_layer_norms_perceiverr N)r! r" r# r$ __doc__r% r4 boolr( r6 r' r8 r9 r; r< r- r r. r/ r1 r1 / sN
%J M4 !! OS s %* d*r. r1 c ^ \ rS rSr% SrSr\\S.rSr \
\ S' Sr \
\ S' S r
\
\ S
' S r\
\ S ' S
r\
\ S' S
r\
\ S' Sr\\
- \ S' Sr\\ S' Sr\\ S' Sr\\ S' Sr\\ S' Sr\\ S' Sr\\ S ' S r\\ S ' Sr \
S - \ S ' S!r \
S - \ S"' S#r \
\ \
- S - \ S$' S%r \\ S&' S!r!\
\ S'' S%r"\\ S(' S r#\\ S)' S*r$\ \%- \ S+' S%r&\\ S,' S r'\\ S-' S*r(\ \%- \ S.' S%r)\\ S/' S r*\+\,- S - \ S0' S r-\+\,- S - \ S1' U 4S2 jr.Sr/U =r0$ )3
IdeficsConfigK a
additional_vocab_size (`int`, *optional*, defaults to 0):
Additional vocabulary size of the model, typically for the special "![]()
" token. Additional vocab tokens
are always trainable whereas regular vocab tokens can be frozen or not.
alpha_initializer (`str`, *optional*, defaults to `"zeros"`):
Initialization type for the alphas.
alphas_initializer_range (`float`, *optional*, defaults to 0.0):
The standard deviation of the truncated_normal_initializer for initializing the alphas in the Gated Cross
Attention.
alpha_type (`str`, *optional*, defaults to `"float"`):
Whether the gating alphas should be vectors or single floats.
cross_layer_interval (`int`, *optional*, default to 1):
Interval for cross attention (from text to image) layers.
qk_layer_norms (`bool`, *optional*, defaults to `False`):
Whether to add layer norm after q and k
freeze_text_layers (`bool`, *optional*, defaults to `True`):
Whether to freeze text layers
freeze_text_module_exceptions (`bool`, *optional*, defaults to `[]`):
Exceptions to freezing text layers when `freeze_text_layers` is `True`
freeze_lm_head (`bool`, *optional*, defaults to `False`):
Whether to freeze lm head
freeze_vision_layers (`bool`, *optional*, defaults to `True`):
Whether to freeze vision layers
freeze_vision_module_exceptions (`bool`, *optional*, defaults to `[]`):
Exceptions to freezing vision layers when `freeze_vision_layers` is `True`
use_resampler (`bool`, *optional*, defaults to `False`):
Whether to use the Resampler
perceiver_config (`IdeficsPerceiverConfig`, *optional*):
Custom perceiver config or dict
Example:
```python
>>> from transformers import IdeficsModel, IdeficsConfig
>>> # Initializing a Idefics idefics-9b style configuration
>>> configuration = IdeficsConfig()
>>> # Initializing a model from the idefics-9b style configuration
>>> model = IdeficsModel(configuration)
>>> # Accessing the model configuration
>>> configuration = model.config
```idefics)perceiver_config
vision_configi }
vocab_sizer additional_vocab_sizei r i + r r r r r dropoutsilur r r zerosalpha_initializeralphas_initializer_ranger,
alpha_typegư> rms_norm_epsT use_cacheN pad_token_id bos_token_id eos_token_idFtie_word_embeddingscross_layer_intervalqk_layer_normsfreeze_text_layersr freeze_text_module_exceptionsfreeze_lm_headfreeze_vision_layers freeze_vision_module_exceptionsr4 rD rC c ~ > U R c [ 5 U l O9[ U R [ 5 ( a [ S0 U R D6U l U R c [ 5 U l O9[ U R [ 5 ( a [ S0 U R D6U l [ TU ] " S0 UD6 g )Nr )rC r1
isinstancedictrD r super
__post_init__)selfkwargs __class__s r/ r` IdeficsConfig.__post_init__ s ($:$ rO rQ rS r) rT rU rV rW rX r* rY rZ r[ r4 rD r^ r rC r` r-
__classcell__)rc s @r/ r@ r@ K s + Z J'=PcdKJ!"3"K "s" s !! GUS[ J #u# $s$&) e) J L% It L#* L#* +,L#S /D(, %% !#! ND ##24!4%<4 ND !%$%46#TE\6 M4 48M4**T187;d --4; ( (r. r@ )r@ r1 r N) r= huggingface_hub.dataclassesr configuration_utilsr utilsr r r1 r@ __all__r r. r/ rk s & " . 3 # 56$* $ 7$$ 56+ - + 7+4 56Y($ Y( 7Y(x Mr.