Zj`SSKrSSKJr SSKJr SSKrSSKJr SSKJ r SSK J r SSK Jr SSKJr SS KJr SS KJr SS KJrJrJr SS KJrJr SS KJr SSKJrJ r J!r!J"r"J#r# SSK$J%r%J&r& SSK'J(r( SSK)J*r* SSK+J,r,J-r- \"S5"SS\R\55r/SFS\R\S\R`S\R`S\R`S\R`S-S\1S\1\2-4Sjjr3"S S!\R\5r4\\!"S"S#9"S$S%\555r5"S&S'\R\5r6"S(S)\R\5r7"S*S+\R\5r8\Rr\/S,.r:"S-S.\5r;"S/S0\R\5r<\!"S1S2\55r=\!"S3S4\=55r>\!"S5S6\55r?"S7S8\R\5r@\\!"S9S#9"S:S;\555rA\!"S\?55rB\!"S?S#9\"S@SA\555rC\!"SBS#9"SCSD\?\55rD/SEQrEg)GN)Callable) dataclass)initialization)ACT2FN)Cache)GenerationMixin)use_kernel_forward_from_hub)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPastBaseModelOutputWithPooling)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringtorch_compilable_check torch_int)can_return_tuplemerge_with_config_defaults)capture_outputs) AutoModel)InternVLConfigInternVLVisionConfigRMSNormcx^\rSrSrS S\SS4U4SjjjrS\RS\R4SjrSr S r U=r $) InternVLVisionRMSNorm-epsreturnNc>[TU]5 [R"[R "U55UlX lg)z4 InternVLVisionRMSNorm is equivalent to T5LayerNorm N)super__init__nn Parametertorchonesweightvariance_epsilon)self hidden_sizer# __class__s /root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/internvl/modeling_internvl.pyr'InternVLVisionRMSNorm.__init__/s/ ll5::k#:; # hidden_statescURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NrT)keepdim) dtypetor*float32powmeanrsqrtr-r,)r.r4 input_dtypevariances r1forwardInternVLVisionRMSNorm.forward7sw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r3c^[URR5SUR3$)Nz, eps=)tupler,shaper-r.s r1 extra_repr InternVLVisionRMSNorm.extra_repr>s* ))*+6$2G2G1HIIr3)r-r,)gư>) __name__ __module__ __qualname____firstlineno__floatr'r*Tensorr@rF__static_attributes__ __classcell__r0s@r1r!r!-sB$$$$$;U\\;ell;JJr3r!modulequerykeyvalueattention_maskscalingdropoutc hUnUn [R"XRSS55U-n UbX-n [RR U SS9n [RR XURS9n [R"X5n U RSS5R5n X4$)Nrrr6dim)ptrainingr) r*matmul transposer( functionalsoftmaxrWr\ contiguous) rQrRrSrTrUrVrWkwargs key_states value_states attn_weights attn_outputs r1eager_attention_forwardrgBsJL<<';';Aq'ABWLL!#4 ==((2(>L==((6??([L,,|:K''1-88:K  $$r3c^\rSrSrSrS\4U4SjjrS S\RS\RS-S\ \ 4S jjr S r U=r $) InternVLVisionAttention\z+Attention Class for InternVL Vision Encoderconfigc$>[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl URnURnSUl [R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR5UlUS:a[R*"U5O[R,"5Ul U(a[/UR5O[R,"5UlU(a[/UR5Ulg[R,"5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gFbiasr)r&r'rkr/ embed_dimnum_attention_heads num_headshead_dim ValueErrorscaleattention_dropoutprojection_dropout use_qk_norm is_causalr(Linearattention_biasq_projk_projv_projprojection_layerDropoutIdentityr!q_normk_norm)r.rk proj_dropoutqk_normr0s r1r' InternVLVisionAttention.__init___s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( !'!9!900 $$ii0NU[UjUjk ii0NU[UjUjk ii0NU[UjUjk " $..$.. I>JQ>N"**\":TVT_T_Ta?F+DNN;BKKM ?F+DNN; BKKM r3Nr4rUrbc pUR5upEnURU5nURU5nURU5n UR U5nUR U5nUR XEURUR5RSS5nUR XEURUR5RSS5nU RXEURUR5RSS5n [R"URR[5n U "UUUU U4UR (dSO UR"UR$SS.UD6upU R XEUR&5n UR)U 5n UR+U 5n X4$)NrrF)rWrVrx)sizer{r|r}rrreshaperqrrr^viewr get_interfacerk_attn_implementationrgr\rurtror~rv)r.r4rUrb batch_sizeseq_len_ query_statesrcrdattention_interfacerfreoutputs r1r@InternVLVisionAttention.forward{s "/!3!3!5 Q{{=1 [[/ {{=1 {{<0 [[, #++JQUQ^Q^_iijkmno '' T^^T]][eefgijk #((dnndmm\ffghjkl (?(M(M KK , ,.E) %8      % $}}C$2H2HJJ %  % ! "))*t~~N &&{3((0##r3)rurkrorrrxrr|rqrvr~rr{rtr}N)rHrIrJrK__doc__rr'r*rMrrr@rNrOrPs@r1riri\sS5Z3Z>/3'$||'$ t+'$+, '$'$r3riz7 Class for outputs of [`InternVLVisionModel`].  custom_introc\rSrSrSrSrg)$InternVLVisionModelOutputWithPoolinga2 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Average of the last layer hidden states of the patch tokens (excluding the *[CLS]* token) if *config.use_mean_pooling* is set to True. If set to False, then the final hidden state of the *[CLS]* token will be returned. N)rHrIrJrKrrNrr3r1rrsr3rcf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)InternVLVisionPatchEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. cH>[TU]5 URURp2URUR pTUSUS-USUS--nUSUS-USUS-4nX lX0lX@lX`lXpl[R"XEX3S9Ul g)Nrr) kernel_sizestride) r&r' image_size patch_size num_channelsr/ num_patches patch_shaper(Conv2d projection) r.rkrrrr/rrr0s r1r'&InternVLVisionPatchEmbeddings.__init__s !'!2!2F4E4EJ$*$7$79K9Kk!!} 1 5*Q-:VW=:XY !!} 1 5z!} ST 7UV $$(&&))L:ir3 pixel_valuesr$cURup#pEX0R:wa [S5eURUR URR R 55nURS5RSS5nU$)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rr) rDrrsrr9r,r8flattenr^)r.rrrheightwidth embeddingss r1r@%InternVLVisionPatchEmbeddings.forwardsz2>2D2D/ & ,, ,w __\__T__5K5K5Q5Q%RS ''*44Q: r3)rrrrrr) rHrIrJrKrr'r*rMr@rNrOrPs@r1rrs. j ELL U\\  r3rc^\rSrSrSrS\SS4U4SjjrS\RS\ S \ S\R4S jr SS \RS \RS-S\R4S jjr Sr U=r$)InternVLVisionEmbeddingsz[ Construct the CLS token, position and patch embeddings. Optionally, also the mask token. rkr$Nc^>[TU]5 [R"[R "SSUR 55UlUR(a<[R"[R "SSUR 55Ul OSUl [U5Ul URUl [UR[R R"5(a UROURUR4UlURR$nUR&(a?[R"[R "SUS-UR 55UlOSUl[R*"UR,5Ulg)Nr)r&r'r(r)r*zerosr/ cls_tokenuse_mask_token mask_tokenrpatch_embeddingsr isinstancer collectionsabcIterabler use_absolute_position_embeddingsposition_embeddingsrhidden_dropout_probrW)r.rkrr0s r1r'!InternVLVisionEmbeddings.__init__s' ekk!Q8J8J&KL   ll5;;q!V=O=O+PQDO"DO =f E ++&++[__-E-EFF   ##V%6%67  ++77  2 2')||EKK;QR?TZTfTf4g'hD $'+D $zz&"<"<= r3rrrcURSS- nURRSS- n[RR 5(dXE:XaX#:Xa UR$URSS2SS24nURSS2SS24nURSnX R S-n X0R S-n [ US-5n URSXU5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU5n[R"Xg4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rNr6r?rrbicubicF)rmode align_cornersrY)rDrr*jit is_tracingrrrpermuter(r_ interpolatercat) r.rrrr num_positionsclass_pos_embedpatch_pos_embedrZ new_height new_widthsqrt_num_positionss r1interpolate_pos_encoding1InternVLVisionEmbeddings.interpolate_pos_encodings]!&&q)A- 0066q9A= yy##%%+*F6?++ +221bqb59221ab59r"q11 __Q// &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr3rbool_masked_poscURu p4nURU5nUR5upxnUbIURR XxS5n UR S5R U 5n USU - -X--nURR USS5n [R"X4SS9nURbX`RXdU5-nURU5nU$)Nr6rrY) rDrrrexpand unsqueezetype_asrr*rrrrW) r.rrrrrrrr mask_tokensw cls_tokenss r1r@ InternVLVisionEmbeddings.forwards +001e**<8 !+!2 Q  &//00bIK))"-55kBA#q1u- ?J^^**:r2> YY 7Q?  # # /#&C&CJX]&^^J\\*- r3)rrWrrrrrr)rHrIrJrKrrr'r*rMintr BoolTensorr@rNrOrPs@r1rrs >3>>,&D5<<&D&DUX&D]b]i]i&DV48ll))D0  r3rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)InternVLVisionMLPi7c>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gr) r&r'rkr hidden_act activation_fnr(ryr/intermediate_sizefc1fc2r.rkr0s r1r'InternVLVisionMLP.__init__8sb  #F$5$5699V//1I1IJ99V55v7I7IJr3r4r$clURU5nURU5nURU5nU$r)rrr)r.r4s r1r@InternVLVisionMLP.forward?s4/ **=9 / r3)rrkrr) rHrIrJrKr'r*rMr@rNrOrPs@r1rr7s)KU\\ellr3r) layer_normrms_normc^\rSrSrSrS\SS4U4SjjrS\RS\ \R\ \R\R4-4Sjr S r U=r $) InternVLVisionLayeriIz?This corresponds to the Block class in the timm implementation.rkr$Nc>[TU]5 URUlSUl[ U5Ul[ U5Ul[UR"URURS9Ul [UR"URURS9Ul URn[R "U["R$"UR5-SS9Ul[R "U["R$"UR5-SS9Ul[R*"UR,5Ulg)Nrr#T) requires_grad)r&r'chunk_size_feed_forward seq_len_dimri attentionrmlpNORM2FN norm_typer/layer_norm_epslayernorm_beforelayernorm_afterlayer_scale_init_valuer(r)r*r+lambda_1lambda_2rrrW)r.rk init_valuesr0s r1r'InternVLVisionLayer.__init__Ls '-'E'E$08$V, '(8(8 9&:L:LRXRgRg h&v'7'789K9KQWQfQfg33  [5::f>P>P3Q%Qaef  [5::f>P>P3Q%Qaef zz&"<"<= r3r4cURURU55up#URU-nX!-nURU5nUR U5nUR U5nUR bUR U-nXA-nU$r)rrrrrrWr)r.r4attention_outputr layer_outputs r1r@InternVLVisionLayer.forward[s#nn  ! !- 0  ==+;;)8 ++M: xx - ||L1 == $==<7L$3 r3) rrrWrrrrrr) rHrIrJrKrrr'r*rMrCr@rNrOrPs@r1rrIsZI >3 > >|| u|| uU\\5<<%?@ @r3rc`^\rSrSrS\SS4U4SjjrS\RS\\ -4Sjr Sr U=r $) InternVLVisionEncoderiwrkr$Nc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) r&r'rkr( ModuleListrangenum_hidden_layersrlayergradient_checkpointing)r.rkir0s r1r'InternVLVisionEncoder.__init__xsS  ]]vOgOgIh#iIhA$7$?Ih#ij &+#$jsA&r4cJURH nU"U5nM [US9$)N)last_hidden_state)r r )r.r4 layer_modules r1r@InternVLVisionEncoder.forward~s.!JJL(7M'+  r3)rkr r ) rHrIrJrKrr'r*rMrCr r@rNrOrPs@r1rrws<,3,,  ||       r3rc^\rSrSr%\\S'SrSrSrSr S/r Sr Sr Sr Sr\\S.r\R&"5U4S j5rS rU=r$) InternVLVisionPreTrainedModelirkinternvl_visionr)imagevideoTr)r4 attentionscF>[TU]U5 [U[5(a|[R "UR 5 URb [R "UR5 URb![R "UR5 gg[U[5(ak[R"URURR5 [R"URURR5 gg)zInitialize the weightsN)r& _init_weightsrrinitzeros_rrrr constant_rrkrr)r.rQr0s r1r+InternVLVisionPreTrainedModel._init_weightss f% f6 7 7 KK(( )  , F--.))5 F6676  3 4 4 NN6??DKK,N,N O NN6??DKK,N,N O5r3r)rHrIrJrKr__annotations__base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_no_split_modules_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backendrri_can_record_outputsr*no_gradrrNrOrPs@r1rrsl  )$O)&*#./N"&--  ]]_ P Pr3rc ^\rSrSrS\SS4U4SjjrSr\\"SS9\ S S \ RS \ RS-S\ \-4S jj555rS rU=r$)InternVLVisionModelirkr$Nc8>[TU]U5 Xl[U5Ul[ U5UlUR(a[R"5O([R"URURS9Ul UR5 g)Nr)r&r'rkrrrencoderuse_mean_poolingr(r LayerNormr/r layernorm post_initrs r1r'InternVLVisionModel.__init__sm   26:,V4 $44BKKM",,vGYGY_e_t_t:u  r3c.URR$r)rrrEs r1get_input_embeddings(InternVLVisionModel.get_input_embeddingss///r3F)tie_last_hidden_statesrrc URXS9nURU5nUSnURU5n[UURUR S9$)z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). )rr)rr4r)rr-r0rr4r)r.rrrbembedding_outputencoder_outputssequence_outputs r1r@InternVLVisionModel.forwards^ ??[TU]5 [R"URR [ SUR- 5S--5Ul[R"URR [ SUR- 5S--URR 5Ul [URUl[R"URR URR 5Ulg)Nrr)r&r'r(r/ vision_configr/rdownsample_ratiorry text_configlinear_1rprojector_hidden_actactlinear_2rs r1r'$InternVLMultiModalProjector.__init__s ,,v';';'G'G#aRXRiRiNiJjnoJo'op  , ,s1v7N7N3N/OST/T TV\VhVhVtVt  &556 &"4"4"@"@&BTBTB`B`a r3cURU5nURU5nURU5nURU5nU$r)rrIrKrL)r.image_featuresr4s r1r@#InternVLMultiModalProjector.forwards@7  m4 /  m4 r3)rKrrIrL) rHrIrJrKrr'r@rNrOrPs@r1rDrDsb~br3rDzM Base class for InternVL outputs, with hidden states and attentions. cB\rSrSr%SrSr\RS-\S'Sr g)InternVLModelOutputWithPastia past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_statesr) rHrIrJrKrrSr* FloatTensorrrNrr3r1rRrRs 59**T18r3rRzx The InternVL model which consists of a vision backbone and a language model, without a language modeling head. cZ^\rSrSrS\4U4SjjrSrSr\\ \ "SS9SS \ RS \ \\ -\\ -S-S \S-S \\S \\-4 Sjj555rS\ R*S\ RS\ R4Sjr\ \ SS\ R*S-S \ RS-S\ R.S-S\ R*S-S\S-S\ RS-S \ \\ -\\ -S-S \S-S \\S \\-4Sjj55rSS\ R.S\4SjjrSrU=r$) InternVLModeli rkc>[TU]U5 [R"UR5Ul[ U5Ul[R"UR5Ul UR5 gr) r&r'r from_configrF vision_towerrDmulti_modal_projectorrHlanguage_modelr1rs r1r'InternVLModel.__init__sY  %11&2F2FG%@%H"'33F4F4FG r3c6URR5$r)r[r4rEs r1r4"InternVLModel.get_input_embeddingss""7799r3c:URRU5 gr)r[set_input_embeddingsr.rTs r1r`"InternVLModel.set_input_embeddingss 007r3zWObtains image last hidden states from the vision tower and apply multimodal projection.rNrvision_feature_layervision_feature_select_strategyrbr$c URURS9nURRnUS:waSUS'UR"S USS.UD6nUS:Xa UR nOUR UnUS:XaUSS2SS2SS24nURSn[US -5n URS n URXU S5nURXuS 9nURU SURS5nURU5nXvl U$) z pixel_values (`torch.FloatTensor]` of shape `(batch_size, channels, height, width)`) The tensors corresponding to the input images. vision_feature_layer (`int` or `list[int]`): Layer index or list of layer indices to extract features from. )r8r6Toutput_hidden_states)r return_dictdefaultNrrr) scale_factorr) r9r8rkrGrYrr4rDrr pixel_shufflerZ pooler_output) r.rrcrdrbrGvision_outputsvision_featureschannels feature_sizers r1get_image_features InternVLModel.get_image_features s1$$TZZ8 ;;77 2 %-1F) ***a RVaZ`a 2 %,>>O,::;OPO )Y 6-aQh7O#((+8S=) $**1- *11*LZ\],,_,\*11*b/BWBWXZB[\44_E'6$r3 input_ids inputs_embedsrOc FUcjX R5"[R"URR[R UR S95:HnURS5nOXRR:HnUR5nURSURS-nURS5RU5RUR 5n[X$R5UR5:HSUSU35 U$)z Obtains multimodal placeholder mask from `input_ids` or `inputs_embeds`, and checks that the placeholder token count is equal to the length of multimodal features. If the lengths are different, an error is raised. )r8devicer6rrz6Image features and image tokens do not match, tokens: z , features: )r4r*tensorrkimage_token_idlongruallsumrDr expand_asr9rnumel)r.rrrsrOspecial_image_maskn_image_tokensn_image_featuress r1get_placeholder_mask"InternVLModel.get_placeholder_maskSs   !.2K2K2M T[[77uzzR_RfRfg3" "4!7!7!; !*kk.H.H!H +//1)//2^5I5I!5LL/99"=GG VYYZgZnZno  - 3 3 59M9M9O OD^DTT`aq`r s "!r3rU position_idsr@c  USLUSL- (a [S5eUcUR5"U5nUbcURUUUSS9Rn U R UR UR 5n URXU S9n URX5nUR"SUUUUS.U D6n [U RU RU RU RUbW S9$SS9$)Nz:You must specify exactly one of input_ids or inputs_embedsT)rrcrdrg)rsrO)rUrr@rs)rr@r4rrSr)rsr4rprkr9rur8rmasked_scatterr[rRrr@r4r) r.rrrrUrr@rsrcrdrbrOr}outputss r1r@InternVLModel.forwardks1  -t"; <YZ Z   557 BM  #!44)%9/M 5 m  ,..}/C/C]EXEXYN!%!:!:~";" *889K\M%% )%+'    +%77#33!//))2>2J   QU   r3rmric UR5up4pVXR-S:wdXB-S:wa [S5eURX4[XR-5[Xb- 55nUR SSSS5R 5nURU[XR-5[XB-5[XbS-- 55nUR SSSS5R 5nU$)aPerform pixel shuffle downsampling on vision features. Args: vision_features (`torch.Tensor`): Input tensor of shape (batch_size, width, height, channels). scale_factor (`float`, *optional*, defaults to `0.5`): Factor by which to downsample. Default is 0.5, which halves the dimensions. Returns: vision_features (`torch.Tensor`): Downsampled tensor of shape (batch_size, height*scale_factor, width*scale_factor, channels/(scale_factor^2)). rzKHeight and width must be divisible by scale_factor for proper downsampling.rrr)rrsrrrra)r.rmrirrrrns r1rjInternVLModel.pixel_shuffles/>.B.B.D+ 6  A %)=)Bjk k*.. s6#893x?V;W *11!Q1=HHJ*.. F12C8L4MsS[mn_nSoOp  *11!Q1=HHJr3)r[rZrYNN)NNNNNNNN)r)rHrIrJrKrr'r4r`rrrr*rTrliststrrrrCrrp LongTensorrrMrrRr@rLrjrNrOrPs@r1rVrV s ~:8 n DH59 ,'',"DIoS 9D@,),d , +, , + + , ,\"))":?:K:K"]b]n]n"0.215.204(,26CG59- ##d*- ''$.-  t+ - &&- -  - ((4/- "DIoS 9D@- ),d - +,-  , ,- - ^!U\\!!!r3rVzT Base class for InternVL causal language model (or autoregressive) outputs. c\rSrSr%SrSr\RS-\S'Sr \RS-\S'Sr \ S-\S'Sr \ \RS-\S'Sr\ \RS-\S'Sr\RS-\S 'S rg) InternVLCausalLMOutputWithPastia loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nlosslogitsr@r4rrSr)rHrIrJrKrrr*rTrrr@rr4rCrrSrNrr3r1rrs &*D%  d ")'+FE   $+$(OUT\(59M5**+d2926Je''(4/648**T18r3rzV The INTERNVL model which consists of a vision backbone and a language model. ch^\rSrSrSS0rS\4U4SjjrSrSrS\ R4S jr \ SS \ RS \\\-\\-S -S \S -S\\S\\-4 Sjj5r\\ SS\ R0S -S \ RS -S\ R2S -S\ R0S -S\S -S\ RS -S \\\-\\-S -S \S -S\ R0S -S\\ R2-S\ R2S -S\\S\\-4Sjj55rSU4SjjrSrU=r$) InternVLForConditionalGenerationizlm_head.weightz(model.language_model.embed_tokens.weightrkc>[TU]U5 [U5Ul[R "UR RUR RSS9Ul UR5 g)NFrm) r&r'rVr>r(ryrHr/ vocab_sizelm_headr1rs r1r')InternVLForConditionalGeneration.__init__sS  "6* yy!3!3!?!?ASASA^A^ejk  r3c6URR5$r)r>r4rEs r1r45InternVLForConditionalGeneration.get_input_embeddingsszz..00r3c:URRU5 gr)r>r`ras r1r`5InternVLForConditionalGeneration.set_input_embeddingss ''.r3r$cUR$r)rrEs r1get_output_embeddings6InternVLForConditionalGeneration.get_output_embeddingss ||r3Nrrcrdrbc BURR"SUUUS.UD6$)N)rrcrdr)r>rp)r.rrcrdrbs r1rp3InternVLForConditionalGeneration.get_image_featuress3zz,, %!5+I    r3rrrUrr@rslabelslogits_to_keep image_sizesc UR"SUUUUUUUUU S. U D6n U Sn[U [5(a [U *S5OU nUR USS2USS245nSnU b3UR "SUXR RRS.U D6n[UUU RU RU RU RS9$)as Example: ```python >>> import torch >>> from transformers import AutoProcessor, AutoModelForImageTextToText >>> torch_device = "cuda" >>> processor = AutoProcessor.from_pretrained("OpenGVLab/InternVL3-1B-hf") >>> model = AutoModelForImageTextToText.from_pretrained( ... "OpenGVLab/InternVL3-1B-hf", dtype=torch.bfloat16, device_map=torch_device ... ) >>> messages = [ ... { ... "role": "user", ... "content": [ ... { ... "type": "image", ... "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg", ... }, ... { ... "type": "image", ... "url": "https://thumbs.dreamstime.com/b/golden-gate-bridge-san-francisco-purple-flowers-california-echium-candicans-36805947.jpg", ... }, ... {"type": "text", "text": "These images depict two different landmarks. Can you identify them?"}, ... ], ... }, ... ] >>> inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(torch_device) >>> generate_ids = model.generate(**inputs, max_new_tokens=200) >>> print(processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)) The images depict the Statue of Liberty and the Golden Gate Bridge. ```) rrrrUrr@rsrcrdrrN)rrr)rrr@r4rrSr)r>rrslicer loss_functionrkrHrrr@r4rrS)r.rrrrUrr@rsrcrdrrrrbrr4 slice_indicesrrs r1r@(InternVLForConditionalGeneration.forwardsh**  %)%+'!5+I#      8B>SV8W8W~ot4]k mA}a,?@A  %%f9P9P9[9[_eD.#33!//)) ' ; ;   r3c z>[T U]"U4UUUUUS.UD6n U(dURSS5(dXIS'U $)N)r@rsrUris_first_iteration use_cacheTr)r&prepare_inputs_for_generationget) r.rrr@rsrrUrrrb model_inputsr0s r1r>InternVLForConditionalGeneration.prepare_inputs_for_generationYsZw<  +'))1    VZZ T%B%B ,8 (r3)rr>r) NNNNNNNNNrN)NNNNNF) rHrIrJrK_tied_weights_keysrr'r4r`r(Modulerrr*rTrrrrrrCrrprrrMrrr@rrNrOrPs@r1rrs +,VW~ 1/ryyDH59  ''  "DIoS 9D@  ),d  +,  + +    .215.204(,26CG59*.-.+/Q ##d*Q ''$.Q  t+ Q &&- Q  Q ((4/Q "DIoS 9D@Q ),d Q   4'Q ell*Q \\D(Q +,Q  / /Q Q l r3r)rr+r=rVr)r)Fcollections.abcrr dataclassesrr*torch.nnr(rr activationsr cache_utilsr generationr integrationsr modeling_layersr modeling_outputsr r rmodeling_utilsrrprocessing_utilsrutilsrrrrr utils.genericrrutils.output_capturingrautorconfiguration_internvlrrrr!rMrLrrgrirrrrr/rrrrr+r=rDrRrVrr__all__rr3r1rs,$! &! )79ddF&ggI5HY'JBIIJ(J6% II% <<% % << % LL4' %  %S[%4F$biiF$R  +E   BII J[ryy[|  3H I+4+\ BII &POPP@& 7& & R 'o ' '"))$  9"9 9  9 l+l  l^  9[9  90 T'>T  Tn r3