ZjMh.SrSSKrSSKrSSKJr SSKJr SSKrSSKJ r SSKJ r SSK J r SS KJr SS KJr SS KJrJrJr SS KJrJr SS KJr SSKJrJrJr SSKJ r J!r!J"r"J#r#J$r$J%r%J&r&J'r'J(r(J)r)J*r* \"SS9\ "SS\555r+\"SS9\ "SS\555r,\"SS9\ "SS\555r-"SS\%5r.\\"SS\555r/\\"SS\555r0"SS \)5r1"S!S"\'5r2"S#S$\!5r3"S%S&\ 5r4"S'S(\#5r5"S)S*\#5r6\"S+S,\&55r7"S-S.\"5r8"S/S0\75r9"S1S2\(5r:"S3S4\*5r;"S5S6\$5r<\"S7S89"S9S:\755r=/S;Qr>g)>> from transformers import CLIPSegTextConfig, CLIPSegTextModel >>> # Initializing a CLIPSegTextConfig with CIDAS/clipseg-rd64 style configuration >>> configuration = CLIPSegTextConfig() >>> # Initializing a CLIPSegTextModel (with random weights) from the CIDAS/clipseg-rd64 style configuration >>> model = CLIPSegTextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```N__name__ __module__ __qualname____firstlineno____doc__AttributeErrorprojection_dim__static_attributes__r$|/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/clipseg/modular_clipseg.pyr"r"/ $%Nr.r"c&\rSrSrSr\"5rSrg)CLIPSegVisionConfigEa Example: ```python >>> from transformers import CLIPSegVisionConfig, CLIPSegVisionModel >>> # Initializing a CLIPSegVisionConfig with CIDAS/clipseg-rd64 style configuration >>> configuration = CLIPSegVisionConfig() >>> # Initializing a CLIPSegVisionModel (with random weights) from the CIDAS/clipseg-rd64 style configuration >>> model = CLIPSegVisionModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```r$Nr%r$r.r/r2r2Er0r.r2c\rSrSr%SrSr\\\\S4-\ S'Sr \\ S'Sr \\ S 'S r \ \-\ S 'S r\\ S 'Sr\\ S'Sr\\ S'Sr\\ S'Srg) CLIPSegConfig[a extract_layers (`list[int]`, *optional*, defaults to `[3, 6, 9]`): Layers to extract when forwarding the query image through the frozen visual backbone of CLIP. reduce_dim (`int`, *optional*, defaults to 64): Dimensionality to reduce the CLIP vision embedding. conditional_layer (`int`, *optional*, defaults to 0): The layer to use of the Transformer encoder whose activations will be combined with the condition embeddings using FiLM (Feature-wise Linear Modulation). If 0, the last layer is used. use_complex_transposed_convolution (`bool`, *optional*, defaults to `False`): Whether to use a more complex transposed convolution in the decoder, enabling more fine-grained segmentation.. Example: ```python >>> from transformers import CLIPSegConfig, CLIPSegModel >>> # Initializing a CLIPSegConfig with CIDAS/clipseg-rd64 style configuration >>> configuration = CLIPSegConfig() >>> # Initializing a CLIPSegModel (with random weights) from the CIDAS/clipseg-rd64 style configuration >>> model = CLIPSegModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # We can also initialize a CLIPSegConfig from a CLIPSegTextConfig and a CLIPSegVisionConfig >>> # Initializing a CLIPSegText and CLIPSegVision configuration >>> config_text = CLIPSegTextConfig() >>> config_vision = CLIPSegVisionConfig() >>> config = CLIPSegConfig(text_config=config_text, vision_config=config_vision) ```)r .extract_layers@ reduce_dimdecoder_num_attention_headsdecoder_attention_dropout quick_geludecoder_hidden_actidecoder_intermediate_sizerconditional_layerF"use_complex_transposed_convolutionr$N)r&r'r(r)r*r9listinttuple__annotations__r;r=r?floatrAstrrBrCrDboolr-r$r.r/r5r5[su! F3M5**C/047>7;Je'',-4;r.rRc\rSrSr%SrSr\RS-\S'Sr \RS-\S'Sr \RS-\S'Sr \RS-\S'Sr \ \S'Sr\\S 'S \\4S jrS rg) CLIPSegImageSegmentationOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Binary cross entropy loss for segmentation. logits (`torch.FloatTensor` of shape `(batch_size, height, width)`): Classification scores for each pixel. conditional_embeddings (`torch.FloatTensor` of shape `(batch_size, projection_dim)`): Conditional embeddings used for segmentation. pooled_output (`torch.FloatTensor` of shape `(batch_size, embed_dim)`): Pooled output of the [`CLIPSegVisionModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPSegVisionModel`]. decoder_output (`CLIPSegDecoderOutput`): The output of the [`CLIPSegDecoder`]. NlossrTconditional_embeddings pooled_outputvision_model_outputdecoder_outputreturncB[SUR555$)Nc3p# UH,n[U[5(aUR5OUv M. g7fN) isinstancer to_tuple).0vs r/ :CLIPSegImageSegmentationOutput.to_tuple..s)^P]1Z;%?%?QZZ\QFP]s46)rGvalues)selfs r/rf'CLIPSegImageSegmentationOutput.to_tuples^PTP[P[P]^^^r.r$)r&r'r(r)r*r\rWrXrHrTr]r^r_r r`rRrGrrfr-r$r.r/rZrZs &*D%  d ")'+FE   $+7;E--4;.2M5$$t+26:3:+/N(/_%*_r.rZc`^\rSrSrSS\R S\R 4U4SjjjrSrU=r $)CLIPSegVisionEmbeddings pixel_valuesrac$>[TU]X5 grdsuperforward)rlrqinterpolate_pos_encoding __class__s r/ruCLIPSegVisionEmbeddings.forwards  ?r.r$T) r&r'r(r)rWrXTensorrur- __classcell__rws@r/roros,@E$5$5@Y^YeYe@@r.roc\rSrSrSrg)CLIPSegTextEmbeddingsr$NrOr$r.r/r~r~rPr.r~c\rSrSrSrg)CLIPSegAttentionr$NrOr$r.r/rrrPr.rc\rSrSrSrg) CLIPSegMLPr$NrOr$r.r/rrrPr.rc\rSrSrSrg)CLIPSegEncoderLayerr$NrOr$r.r/rrrPr.rcp\rSrSrSrS\R S\R S\\R4Sjr Sr g) CLIPSegDecoderLayerz CLIPSeg decoder layer, which is identical to `CLIPSegEncoderLayer`, except that normalization is applied after self-attention/MLP, rather than before. rUattention_maskrac UnUR"SUUS.UD6upXA-nURU5nUnURU5nXA-nURU5nU$)N)rUrr$) self_attn layer_norm1mlp layer_norm2)rlrUrkwargsresidual_s r/ruCLIPSegDecoderLayer.forwardsz !>> ')   !0 ((7  /  0 ((7 r.r$N) r&r'r(r)r*rWrzrGrXrur-r$r.r/rrs; ||  u  ! r.rcP\rSrSr\\/\S.r\R"5S5r Sr g)CLIPSegPreTrainedModel)rUrVc URRn[U[5(a[R "UR RSUS-S9 [R "URRSUS-S9 [R"UR[R"URRS5RS55 GOY[U[5(Ga[R "UR SUR"S-U-S9 [R "UR$RURR&U-S9 [R "URRURR&U-S9 [R"UR[R"UR(5RS55 GOC[U[*5(aUR"S-SURR,-S--U-nUR"S-U-n[R "UR.RUS9 [R "UR0RUS9 [R "UR2RUS9 [R "UR4RUS9 GOG[U[65(aURR8S-SURR,-S--U-nSURR8-S-U-n[R "UR:RUS9 [R "UR<RUS9 O[U[>5(ar[R "UR@RURBS-U-S9 [R "URDRURFS-U-S9 [U[HRJ5(a@[RL"URN5 [RP"UR5 [U[HRR5(a/URNb![RL"URN5 g g g ) zInitialize the weightsr>g{Gz?)meanstd)rg)rrN)*configinitializer_factorrer~initnormal_token_embeddingweightposition_embeddingcopy_ position_idsrWarangeshapeexpandroclass_embedding embed_dimpatch_embeddinginitializer_range num_positionsrnum_hidden_layersq_projk_projv_projout_projr hidden_sizefc1fc2 CLIPSegModeltext_projectiontext_embed_dimvisual_projectionvision_embed_dimr LayerNormzeros_biasones_Linear)rlmodulefactor in_proj_std out_proj_stdfc_stds r/ _init_weights$CLIPSegPreTrainedModel._init_weightssi// f3 4 4 LL//66Sftm T LL2299RV W JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h  7 8 8 LL//cv?O?OQU?UX^?^ _ LL//66FMM<[<[^d>[TU]U5 URUl[R"UR UR 5Ul[R"UR UR 5UlUR(aURRS-URRS-4n[R"[R"UR UR SSS9[R"5[R"UR UR S-USUSS9[R"5[R"UR S-SUSUSS95UlON[R"UR SURRURRS9Ul[#UR$5n[R&"[)U5Vs/sH8n[R"URR*UR 5PM: sn5Ul[.R0"UR5nUR UlUR2UlUR6UlS Ul[R&"[)[#UR$55Vs/sHn[=U5PM sn5UlURA5 gs snfs snf) Nr<rr) kernel_sizepaddingrr)rstride)rrelu)!rt__init__rCrrr,r;film_mulfilm_addrD vision_config patch_size SequentialConv2dReLUConvTranspose2dtransposed_convolutionlenr9 ModuleListrangerreducescopydeepcopyr=num_attention_headsrBintermediate_size hidden_actrlayers post_init)rlrtransposed_kernelsdepthrdecoder_configrws r/rCLIPSegDecoder.__init__*sg  !'!9!9 &"7"79J9JK  &"7"79J9JK  4 4"("6"6"A"AQ"FH\H\HgHgklHl!m *,-- &++V->->AWXY ""%%%%* 21 5-a0   ""%%*A;Ma;PYklmYn +D '+-*<*<!!1f&:&:&E&EfNbNbNmNm+D 'F))*}}UZ[`Ua bUaPQRYYv++779J9J KUa b  v';';<%+%6%6"-3-O-O*+1+K+K($*!mmRWX[\b\q\qXrRs$tRsQ%8%HRs$tu   c%us ?L#LrUr]rrac USSS2nSn[[X@RUR55Hunupxn Ub U "U5U-nOU "U5nX`R:XaJUR U5UR SSS5-URU5-nUR SSS5nU"U4SS0UD6nM USS2SS2SS24RSS5n[[R"URS55n URSn URXRSX5nURU5RS5n [!U S9$)a conditional_embeddings (`torch.FloatTensor` of shape `(batch_size, config.projection_dim)`, *optional*): The conditional embeddings for the query images. If provided, the model will use this instead of computing the embeddings from the conditional_pixel_values. Nrrrrr)rT) enumerateziprrrCrpermuter transposerFmathsqrtrviewrsqueezerR) rlrUr]r activationsoutputi activationlayerreducesize batch_sizerTs r/ruCLIPSegDecoder.forwardVs[$DbD) .7KVZVbVb8c.d *A* 6! +f4 +***'=>PQSTVWAXX[_[h[h*\ 1a06A$A&AF/e12q!++Aq1499V\\!_-.+11!4 Za$E,,V4<[TU]"S0UD6$)a Examples: ```python >>> from transformers import AutoTokenizer, CLIPSegTextModel >>> tokenizer = AutoTokenizer.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegTextModel.from_pretrained("CIDAS/clipseg-rd64-refined") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```r$rsrl super_kwargsrws r/ruCLIPSegTextModel.forwards w...r.r$) r&r'r(r)rGr rur-r{r|s@r/rrs/1K)K//r.rc n^\rSrSrS S\R S-S\S-S\\S\ \ -4U4Sjjjr Sr U=r $) CLIPSegVisionModelirqNrvrrac &>[TU]"X40UD6$)a Examples: ```python >>> import httpx >>> from io import BytesIO >>> from PIL import Image >>> from transformers import AutoProcessor, CLIPSegVisionModel >>> processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegVisionModel.from_pretrained("CIDAS/clipseg-rd64-refined") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rsrlrqrvrrws r/ruCLIPSegVisionModel.forwards8w|PPPr.r$ry)r&r'r(r)rWrXrKr r rGr rur-r{r|s@r/rrsY15Q''$.Q#'+Q+, Q + + QQr.rc ^\rSrSrU4SjrS S\R S\S\\ S\ \ -4U4Sjjjr S S\4U4Sjjjr S rU=r$) ric $>[TU]"S0UD6$)a Examples: ```python >>> import torch >>> from transformers import AutoTokenizer, CLIPSegModel >>> tokenizer = AutoTokenizer.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegModel.from_pretrained("CIDAS/clipseg-rd64-refined") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... text_features = model.get_text_features(**inputs) ```r$)rtget_text_featuresrs r/r CLIPSegModel.get_text_featuressw(8<88r.rqrvrrac &>[TU]"X40UD6$)a$ Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPSegModel >>> from transformers.image_utils import load_image >>> processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegModel.from_pretrained("CIDAS/clipseg-rd64-refined") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... image_features = model.get_image_features(**inputs) ```)rtget_image_featuresrs r/rCLIPSegModel.get_image_featuress2w),[TZ[[r.c *>[TU]"SSU0UD6 g)al return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPSegModel >>> from transformers.image_utils import load_image >>> processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegModel.from_pretrained("CIDAS/clipseg-rd64-refined") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```rvNr$rs)rlrvrrws r/ruCLIPSegModel.forwards6 Z1IZ\Zr.r$ry)r&r'r(r)r rWrXrKr r rGr rrur-r{r|s@r/rrsi9(*.\''\#'\+, \ + + \\6[[[r.rzn CLIPSeg model with a Transformer-based decoder on top for zero-shot and one-shot image segmentation. ) custom_introc ^\rSrSr%\\S'S\4U4SjjrSS\S-S\RS-S\RS-S\RS-S \RS-S \R4 S jjr \ \ SS\RS-S \RS-S \RS-S \RS-S\RS-S\RS-S\RS-S\S\\S \\-4Sjj55rSrU=r$)CLIPSegForImageSegmentationirc>[TU]U5 [U5UlURUl[ U5UlUR5 grd)rtrrclipr9rdecoderr)rlrrws r/r$CLIPSegForImageSegmentation.__init__s@   ( $33%f-  r.Nr input_idsrrconditional_pixel_valuesracUb^[U5U:wa [S5e[R"5 URR X#US9R nSSS5 U$Ub_[U5U:wa [S5e[R"5 URRU5R nSSS5 U$[S5e!,(df  W$=f!,(df  W$=f)Nz@Make sure to pass as many prompt texts as there are query images)rrzAMake sure to pass as many prompt images as there are query imagesz[Invalid conditional, should be either provided as `input_ids` or `conditional_pixel_values`)r ValueErrorrWrrr  pooler_outputr)rlrrrrrr]s r/get_conditional_embeddings6CLIPSegForImageSegmentation.get_conditional_embeddingss  9~+ !cdd)-)D)D<*E*-'!&%& 1+, : !dee)-)E)EF^)_)m)m&!&% m !&%!&%s%C&C! C! C0rqr]labelsrvrc h[R"5 SU S'URR"S UUS.U D6n U Rn U R n UR V s/sH oU S-PM nn [U RU RU R U RS9n SSS5 Uc!URURSUUUUS9nO]URSURS:wa [S 5eURSURR:wa [S 5eUR"WU40U D6nUR nSnUb9UR#UR$5n[&R("5nU"UU5n[+UUUW W US 9$s sn f!,(df  GN=f) a conditional_pixel_values (`torch.FloatTensor`, *optional*): The pixel values of the conditional images. conditional_embeddings (`torch.FloatTensor` of shape `(batch_size, config.projection_dim)`, *optional*): The conditional embeddings for the query images. If provided, the model will use this instead of computing the embeddings from the conditional_pixel_values. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPSegForImageSegmentation >>> from transformers.image_utils import load_image >>> processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") >>> model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> texts = ["a cat", "a remote", "a blanket"] >>> inputs = processor(text=texts, images=[image] * len(texts), padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> logits = outputs.logits >>> print(logits.shape) torch.Size([3, 352, 352]) ```Toutput_hidden_states)rqrvr)last_hidden_staterrUrVNr)rrrrrzWMake sure to pass as many conditional embeddings as there are query images in the batchzcMake sure that the feature dimension of the conditional embeddings matches `config.projection_dim`.)r\rTr]r^r_r`r$)rWrrrrrUr9r r#rVrrrrr,rrTtodevicerBCEWithLogitsLossrZ)rlrrqrr]rrr rvrvision_outputsr^rUrrdecoder_outputsrTr\loss_fns r/ru#CLIPSegForImageSegmentation.forward-sb]]_-1F) *!YY99))AN +88M*88M9=9L9LM9LAQ/9LKM8"0"B"B,::,::)44 N, " )%)%D%D'--a0#-))A &E& "&++A.,2D2DQ2GG m&++A.$++2L2LL 0 ,,  "   !''  YYv}}-F**,G66*D-#9' .*   [N_sA F"!F26F"F"" F1)rrr9)NNNNN)NNNNNNNT)r&r'r(r)r5rHrrFrWrzrXrrr LongTensorrKr r rGrMrur-r{r|s@r/rrs }"&)-.2,08< &$J&<<$&& t+ & llT) & #(,,"5 &   &:/315=A;?.204*.)-n $$t+n ''$.n #("3"3d": n !& 1 1D 8 n  t+ n &&-n   4'n #'n +,n   n n r.r)r5r"r2rrrrr)?r*rr dataclassesrtypingrrWhuggingface_hub.dataclassesrrrrmodeling_outputsr processing_utilsr utilsr r r utils.genericrrutils.output_capturingrclip.configuration_cliprrrclip.modeling_cliprrrrrrrrrrrr"r2r5rMrRrZror~rrrrrrrrrrr__all__r$r.r/r8s ! .&:&DDI5RR    /0&&1&(/0&*&1&(/0+5J+51+5\ J  <;< <" _[_ _6@2@  .  }    * *>-%0-%-%` [ U3+U3p/}/(QQ@H[9H[V X "8X  X v r.