Zjh.XSSKrSSKJr SSKJrJrJrJrJr SSKJ r SSK J r SSK J r Jr SSKJr SS KJrJrJr SS KJr S S KJr \R2"\5r"S S\5r"SS\5r"SS\R<5r"SS\5r "SS\5r!"SS\5r"/SQr#g)N)nn)LlavaCausalLMOutputWithPastLlavaForConditionalGeneration LlavaModelLlavaModelOutputWithPastLlavaPreTrainedModel)ACT2FN)Cache)BaseModelOutputWithPastBaseModelOutputWithPooling)Unpack)TransformersKwargsauto_docstringlogging)can_return_tuple)VipLlavaConfigc\rSrSrSrg)VipLlavaModelOutputWithPast&N__name__ __module__ __qualname____firstlineno____static_attributes__r~/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/vipllava/modular_vipllava.pyrr&rrc\rSrSrSrg)VipLlavaCausalLMOutputWithPast*rNrrrr r#r#*r!rr#c6^\rSrSrS\4U4SjjrSrSrU=r$)VipLlavaMultiModalProjector.configcB>[TU]5 [UR[5(aSO[ UR5n[ R"X!RR-URS9Ul [ R"X!RR-URRSS9Ul[UR Ul[ R"URRURRSS9Ulg)Nr)epsT)bias)super__init__ isinstancevision_feature_layersintlenr LayerNorm vision_config hidden_sizeprojector_layernorm_epsprojector_layernormLinear text_configlinear_1r projector_hidden_actactlinear_2)selfr(num_feature_layers __class__s r r-$VipLlavaMultiModalProjector.__init__/s ",V-I-I3"O"OQUXY_YuYuUv#%<< !5!5!A!A AvGeGe$   !5!5!A!A A    * *  &556 &"4"4"@"@&BTBTB`B`gkl rcURU5nURU5nURU5nURU5nU$N)r6r9r;r<)r= hidden_statess r forward#VipLlavaMultiModalProjector.forward>sB00?  m4 /  m4 r)r;r9r<r6) rrrrrr-rDr __classcell__)r?s@r r&r&.s m~ mrr&c\rSrSrSrg)VipLlavaPreTrainedModelFrNrrrr rHrHFr!rrHc\rSrSr\\"SS9SS\RS\\ \-S-S\ \ S\ \ -4S jj55r\\SS \RS-S\RS-S \R S-S \RS-S \S-S\RS-S\\ \-S-S\S-S\ \ S\ \-4Sjj55rSrg) VipLlavaModelJzWObtains image last hidden states from the vision tower and apply multimodal projection.) custom_introN pixel_valuesr/kwargsreturnc xUbUOURRnSUS'UR"U40UD6n[U[5(aUR USS2SS24nON33NC&4# ls&!B7 input_idsattention_mask position_idspast_key_values inputs_embeds use_cache lm_kwargsc UbUOURRnUSLUSL- (a [S5eUcUR5"U5nUb`UR X'S9R n U R URUR5n URXU S9n URX5nUR"SUUUUUS.U D6n [U RU RU RU R UbW OSS9n U $)z vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. Nz:You must specify exactly one of input_ids or inputs_embedsrNr/)rdr\)rarbrcrdre)last_hidden_statercrC attentionsimage_hidden_statesr)r(r/ ValueErrorget_input_embeddingsr^rZtodevicedtypeget_placeholder_maskmasked_scatterlanguage_modelrrircrCrj)r=r`rNrarbrcrdr/rerfr\special_image_maskoutputsoutputs r rDVipLlavaModel.forwardss;(&;%F !DKKLmLm   -t"; <YZ Z   557 BM  #!44)5m ,..}/C/C]EXEXYN!%!:!:~";" *889K\M+/+>+>, )%+' ,  , -%77#33!//))2>2JPT   rrrB)NNNNNNNN)rrrrrrrW FloatTensorr0listrrtupler r^ LongTensorTensorr boolrrDrrrr rKrKJs]n 9="''" #T#Y5"+, " + + ""H.215.204(,268URR"SXS.UD6$)rRrhr)modelr^)r=rNr/rOs r r^3VipLlavaForConditionalGeneration.get_image_featuress+zz,, % V\  rr`rarbrcrdlabelsrelogits_to_keeprfc UbUOURRnUR"SUUUUUUU US.U D6n U Rn [ U [ 5(a [ U *S5OU nURU SS2USS245nSnUb.URXURRRS9n[UUU RU RU RU RS9$)a vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, VipLlavaForConditionalGeneration >>> model = VipLlavaForConditionalGeneration.from_pretrained("llava-hf/vip-llava-7b-hf", device_map="auto", dtype=torch.float16) >>> processor = AutoProcessor.from_pretrained("llava-hf/vip-llava-7b-hf") >>> prompt = "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.###Human: \n{}###Assistant:" >>> question = "Can you please describe this image?" >>> prompt = prompt.format(question) >>> url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/compel-neg.png" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(text=text, images=image, return_tensors="pt").to(0, torch.float16) >>> # Generate >>> generate_ids = model.generate(**inputs, max_new_tokens=20) >>> processor.decode(generate_ids[0][len(inputs["input_ids"][0]):], skip_special_tokens=True) The image features a brown and white cat sitting on a green surface, with a red ball in its ```N)r`rNrarbrcrdrer/)logitsr vocab_size)lossrrcrCrjrkr)r(r/rrir.r0slicelm_head loss_functionr8rr#rcrCrjrk)r=r`rNrarbrcrdr/rrerrfrurC slice_indicesrrs r rD(VipLlavaForConditionalGeneration.forwardsj&;%F !DKKLmLm 04zz 0 %)%+'"7 0  0  11 8B>SV8W8W~ot4]k mA}a,?@A  %%Vt{{OfOfOqOq%rD-#33!//)) ' ; ;   rrrB) NNNNNNNNNr)rrrrrrWrxr0ryrrrzr r^rr{r|r r}r#rDrrrr rrs9= ''  #T#Y5 +,  + +   ".215.204(,268<*.!%-.R ##d*R ''$.R  t+ R &&- R  R ((4/R  #T#Y5R   4'R $;R ell*R ./R  / /R R rr)rKrrH)$rWr(transformers.models.llava.modeling_llavarrrrr activationsr cache_utilsr modeling_outputsr r processing_utilsrutilsrrr utils.genericrconfiguration_vipllavar get_loggerrloggerrr#Moduler&rHrKr__all__rrr rs " S&@@-2   H % ":  %@ "))0 2 `J`Fg 'Dg T [r