Zj h,SSKrSSKJr SSKJrJrJr SSKrSSK r SSK J r SSK J r JrJr SSKJrJrJrJr SSKJrJrJr SS KJr SS KJr \(aSS KJr "S S \SS9rS\4Sjr Sr!Sr"Sr#Sr$\"SS9\"SS\555r%S/r&g)N) accumulate) TYPE_CHECKINGOptionalUnion) BatchFeature) ImageInputis_valid_image load_image)MultiModalDataProcessingKwargsProcessorMixinUnpack) AddedToken BatchEncoding TextInput)auto_docstring)requires)PreTokenizedInputc0\rSrSrSS0SSSS.SS0S .rS rg ) ColModernVBertProcessorKwargs(paddinglongestTchannels_first)return_row_col_info data_formatdo_convert_rgbreturn_tensorspt) text_kwargs images_kwargs common_kwargsN)__name__ __module__ __qualname____firstlineno__ _defaults__static_attributes__r$ڍ/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/colmodernvbert/processing_colmodernvbert.pyrr(s/ y $(+" +D1 Ir+rF)totalreturncR[U[5=(a URS5$)Nhttp) isinstancestr startswith)vals r,is_urlr56s c3  :CNN6$::r+c<[U5=(d [U5$N)r5r )elems r,is_image_or_image_urlr9:s $< />$//r+c Sn[U5H7n[U5H nUUSUS-SUS-S3-UU--- nM" US- nM9 USU3U-UU--U-- nU$)zKPrompt with expanded image tokens for when the image is split into patches. )range) image_seq_len image_rows image_colsfake_token_around_image image_tokenglobal_img_tokentext_split_imagesn_hn_ws r,_prompt_split_imagerK>sZ $C *+sQwiuS1WIQ/OOU`TaerRrr  % T! ! $ %&   M] * +% % ' r+c&UU-UU--U-$)z5Prompt with expanded image tokens for a single image.r$)rBrErFrGs r,_prompt_single_imagerMQs6 # #   M] * +% % 'r+cLUS:XaUS:Xa [UUUUS9$[X XXE5$)Nr)rErFrG)rMrK)rCrDrBrErFrGs r,get_image_prompt_stringrO[s@Q:?# $;#-   :  r+)torch)backendsc ^\rSrSrSrSS\S\S-S\S-4U4SjjjrSr\ S S \ \ \ -\ \ \ -S \ \ S \ \ \ S 4S\S-S \\S \4 Sjj5rS\ S\ \S \ \ \4SjrS!SjrS!S \ S-S \\S \4SjjrS \ \ \ -S \\S \4SjrS"S\ S\ S4S\ S\ S4S\S\SS\ S\4S S4 SjjrSrU=r$)#ColModernVBertProcessorja Constructs a ColModernVBert processor which wraps a ModernVBertProcessor and special methods to process images and queries, as well as to compute the late-interaction retrieval score. [`ColModernVBertProcessor`] offers all the functionalities of [`ModernVBertProcessor`]. See the [`~ModernVBertProcessor.__call__`] for more information. Args: image_processor ([`Idefics3ImageProcessor`]): An instance of [`Idefics3ImageProcessor`]. The image processor is a required input. tokenizer (`PreTrainedTokenizerFast`, *optional*): An instance of [`PreTrainedTokenizerFast`]. This should correspond with the model's text model. The tokenizer is a required input. image_seq_len (`int`, *optional*, defaults to 64): The length of the image sequence i.e. the number of tokens per image in the input. visual_prompt_prefix (`Optional`, *optional*): A prefix to be prepended to visual prompts. query_prefix (`Optional`, *optional*): A prefix to be prepended to query prompts. NrBvisual_prompt_prefix query_prefixc >Sn[SSSS9RUl[SSSS9RUl[SSSS9RUlSUlX@lURUR5UlURUR5Ul URUR 5Ul [S 5VV s/sH3n[S 5H oRS US -S U S -S 35PM" M5 sn nUl [R"S5UlSURURUR/0n UR!U 5 URUR5Ul["T U]H"X4SU0UD6 U=(d SURS3UlU=(d SUlURUlgs sn nf)aJ image_seq_len (`int`, *optional*, defaults to 64): The length of the image sequence i.e. the number of tokens per image in the input. visual_prompt_prefix (`str`, *optional*): A string that gets tokenized and prepended to the image tokens. query_prefix (`str`, *optional*): A prefix to be used for the query. NzFT) normalizedspecialzzz r<r=r>r?z*(\n?\n?|\n?)+additional_special_tokens chat_templatez<|begin_of_text|>User:z0Describe the image. Assistant:r;)rcontentfake_image_tokenrFend_of_utterance_tokenglobal_image_tagrBconvert_tokens_to_idsimage_token_idfake_image_token_idglobal_image_token_idrA row_col_idsrecompile%_regex_to_remove_extra_special_tokensadd_special_tokenssuper__init__rUrVquery_augmentation_token) selfimage_processor tokenizerr\rBrUrVkwargsij tokens_to_add __class__s r,rk ColModernVBertProcessor.__init__|s$ *+FSXbf g o o%iE4PXX&01ERWae&f&n&n# .*'==d>N>NO#,#B#B4CXCX#Y %.%D%DTEZEZ%["SXYZS[ S[aejklem`a + +eAE7%Awa,H Iem IS[  68ZZ@m5n2 (%%  ++*  $$]3'==d>N>NO [=[TZ[$8% $T%5%5$66g h !).B(,(C(C%1 s :Gc/nUHnn/nUHRn[U5(aURU5 M&[U5(dM8UR[U55 MT URU5 Mp U$r7)r appendr5r )rmprompts prompt_imagespromptimagesr8s r,_extract_images_from_prompts4ColModernVBertProcessor._extract_images_from_promptssi FF!$''MM$'D\\MM*T"23    (r+r{textrrpr.c v UcUc [S5eUR"[4SURR0UD6nUbUO UR nUSR SS5nUSR SS5n/n/n 0n Ub{[U[5(aU/nO8[U[5(d#[US[5(d [S 5eUV s/sHoRUR5PM nn UGb[U5(aU//nGO,[U[[45(a[US5(aUb[U5[U5:wa>[S URS [U5S URS [U5S3 5eS/[[!U55-n [#[U55V s/sHn XU XS-PM nn O^U/nOZ[U[[45(d?[US[[45(d![USS5(d [S5eUV s/sHn [U 5PM n n UV Vs/sH0oVs/sH n[%U5(a ['U5OUPM" snPM2 nn nUR("U40USD6nU R+U5 UGbX:wa[SUSU S35eU R SUVs/sH nS/U-PM sn5nU R SUVs/sH nS/U-PM sn5nUR,nURnUR.n/n/n[1UUU5Hun nn/n/nU5H^unn[3UUUUUUS9nUR S-U-S-nUR5UR S-UU--5 UR5U5 M` UR5U5 U R7U5n [U 5S:Xa [S5eU Sn [9U5Hun nU UU U S--- n M UR5U 5 M UR"U40USD6n!UR;UU!S/S9 U R+U!5 O`Ub][=U5(a%[S[U5S URS35eUR"S#SU0USD6n!U R+U!5 U(aUR?U S W5U S!'[AXS"9$s sn fs sn fs sn fs snfs snn fs snfs snf)$z image_seq_len (`int`, *optional*): The length of the image sequence. If not provided, the default value of self.image_seq_len is used. image_seq_len should be equal to int(((image_size // patch_size) ** 2) / (scale_factor**2)) Nz+You must provide either `text` or `images`.tokenizer_init_kwargsr!return_mm_token_type_idsFrrzAInvalid input text. Please provide a string, or a list of stringszThe total number of zP tokens in the prompts should be the same as the number of images passed. Found  z tokens and z images.r=zdInvalid input images. Please provide a single image or a list of images or a list of list of images.r"z!The number of images in the text z and images z should be the same.rowscols)rFrErGrz.The image token should be present in the text.image) modalitieszFound z. tokens in the text but no images were passed.r~ input_idsmm_token_type_ids)data tensor_typer$)! ValueError _merge_kwargsrro init_kwargsrBpopr1r2listcountrFr9tuplesumlenrrAr5r rnupdater^r`ziprOrwsplit enumerate_check_special_mm_tokensanycreate_mm_token_type_idsr)"rmr{r~rBrp output_kwargsrrn_images_in_textn_images_in_imagesinputssamplecumsum_images_in_textrqim image_inputsn_imagesrCrDr^rFrGprompt_stringsbatch_image_seq_lengths sample_rows sample_colsimage_prompt_stringsimage_seq_lengthsn_rowsn_colsimage_prompt_string row_length split_sample text_inputss" r,__call__ ColModernVBertProcessor.__call__s T R  $V,,!(FT5M227LVTUY7W7W#+,F ;(243C3C2DE&&)*:&;%N>N=O|\_`f\g[hhpr ./C$zBR7S2T,T)"'s+;'22R7FSW!WRV"3"3e";f"DRV !Wmm"'%m4$ "Xs*Cquery_embeddingsz torch.Tensorpassage_embeddings batch_size output_dtypez torch.dtype output_devicez torch.devicec [U5S:Xa [S5e[U5S:Xa [S5eUSRUSR:wa [S5eUSRUSR:wa [S5eUcUSRn/n[ S[U5U5GHn/n[ R RRRXXs-SSS9n [ S[U5U5H}n [ R RRRX*X-SSS9n UR[ R"SX5RS S 9SRS S 95 M UR[ R"US S 9RU5RU55 GM [ R"USS 9$) a Compute the late-interaction/MaxSim score (ColBERT-like) for the given multi-vector query embeddings (`qs`) and passage embeddings (`ps`). For ColQwen2, a passage is the image of a document page. Because the embedding tensors are multi-vector and can thus have different shapes, they should be fed as: (1) a list of tensors, where the i-th tensor is of shape (sequence_length_i, embedding_dim) (2) a single tensor of shape (n_passages, max_sequence_length, embedding_dim) -> usually obtained by padding the list of tensors. Args: query_embeddings (`Union[torch.Tensor, list[torch.Tensor]`): Query embeddings. passage_embeddings (`Union[torch.Tensor, list[torch.Tensor]`): Passage embeddings. batch_size (`int`, *optional*, defaults to 128): Batch size for computing scores. output_dtype (`torch.dtype`, *optional*, defaults to `torch.float32`): The dtype of the output tensor. If `None`, the dtype of the input embeddings is used. output_device (`torch.device` or `str`, *optional*, defaults to "cpu"): The device of the output tensor. Returns: `torch.Tensor`: A tensor of shape `(n_queries, n_passages)` containing the scores. The score tensor is saved on the "cpu" device. rzNo queries providedzNo passages providedz/Queries and passages must be on the same devicez-Queries and passages must have the same dtypeT) batch_first padding_valuez bnd,csd->bcnsr)dimrr=)rrdevicedtyperArPnnutilsrnn pad_sequencerweinsummaxrcatto) rmrrrrrscoresrq batch_scores batch_queriesrrbatch_passagess r,score_retrieval'ColModernVBertProcessor.score_retrievals@  A %23 3 ! "a '34 4 A  % %);A)>)E)E ENO O A  $ $(:1(=(C(C CLM M  +A.44L%'q#./:\]"A"##LL-PTTYZT[\]^bbghbi C MM%))La8;;LILL][ \=yyQ''r+) rhr_r^rcr`rdrBrFrbrlrVrerU)NN@NN)NNNr7)Ncpu)r%r&r'r(__doc__intr2rkr|rr rrrrrrrrrrrrrrr* __classcell__)rts@r,rSrSjs $+/#'3D  3D "Dj 3DDj3D3Dj JNbf$( @ET*--T*5E0FF@EI2DOTJ]E^^_@ETz @E 67 @E  @E@ED!$!QUVYQZ!_cdhildm_n!*#-N%)@T!@67@  @D7$y/)7677  7z0449 >(^0D DE>(".$~2F"FG>( >( }- >( ^S01 >( >(>(r+rS)'rf itertoolsrtypingrrrnumpyrrPfeature_extraction_utilsr image_utilsr r r processing_utilsr r rrtokenization_utils_baserrrrrutils.import_utilsrrrboolr5r9rKrMrOrS__all__r$r+r,r s* 11 4AAXXKK#*< $4E ;4;0&  :G(nG(G(T % %r+