ZjSSKrSSKJr SSKJr SSKJr SSKJ r J r SSK J r SSK JrJrJr SS KJrJrJrJr "S S \5r\"S S \55r"SS\\5r\"SS9"SS\\55r/SQrg)N) IJepaConfig)initialization)BaseModelOutputWithPoolingImageClassifierOutput)Unpack)TransformersKwargsauto_docstring torch_int) ViTEmbeddingsViTForImageClassificationViTModelViTPreTrainedModelc ^\rSrSrSS\S\SS4U4SjjjrS\RS\ S \ S\R4S jr SS \RS \RS-S \S\R4Sjjr Sr U=r$)IJepaEmbeddings configuse_mask_tokenreturnNc>[TU]X5 U?URRn[ R "[R"SX1R55Ul g)N) super__init__ cls_tokenpatch_embeddings num_patchesnn Parametertorchrandn hidden_sizeposition_embeddings)selfrrr __class__s x/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/ijepa/modular_ijepa.pyrIJepaEmbeddings.__init__sH 0 N++77 #%<< A{L^L^0_#` embeddingsheightwidthc,URSnURRSn[RR 5(dXE:XaX#:Xa UR$URnURSnX R -nX0R -n [ US-5n URSXU5nURSSSS5n[RRUX4SSS 9nURSSSS5RSSU5nU$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rg?rrr bicubicF)sizemode align_corners) shaper#r jit is_tracing patch_sizer reshapepermuter functional interpolateview) r$r)r*r+r num_positionspatch_pos_embeddim new_height new_widthsqrt_num_positionss r&interpolate_pos_encoding(IJepaEmbeddings.interpolate_pos_encodings!&&q) 0066q9 yy##%%+*F6?++ +22r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nr( pixel_valuesbool_masked_posrAcnURupEpgURXS9nUbXURSn URRXIS5n UR S5R U 5n USU - -X--nU(aXR XU5-nOXR-nURU5nU$)N)rArr-g?) r2r mask_tokenexpand unsqueezetype_asrAr#dropout) r$rCrDrA batch_size_r*r+r) seq_length mask_tokensmasks r&forwardIJepaEmbeddings.forward<s (4'9'9$ v**<*k  &#))!,J//00LK",,R088ED#sTz2[5GGJ $#&C&CJX]&^^J#&>&>>J\\*- r()r#)F)NF)__name__ __module__ __qualname____firstlineno__rboolrr TensorintrA BoolTensorrP__static_attributes__ __classcell__r%s@r&rr sa{aDaTaa%5<<%%UX%]b]i]i%T48). ll))D0#'   r(rc\rSrSr\R "5S\R\R-\R-SS4Sj5r Sr g)IJepaPreTrainedModelWmodulerNc[U[R[R45(ac[R "UR SURRS9 URb![R"UR5 gg[U[R5(aA[R"UR5 [R"UR 5 g[U[5(ac[R "URSURRS9 URb![R"UR5 ggg)zInitialize the weightsg)meanstdN) isinstancerLinearConv2dinit trunc_normal_weightrinitializer_rangebiaszeros_ LayerNormones_rr#rF)r$r`s r& _init_weights"IJepaPreTrainedModel._init_weightsYs fryy"))4 5 5   v}}3DKK ]]_ /BII $9BLL$H /T / /r(r^c<^\rSrSrSS\S\S\4U4SjjjrSrU=r$) IJepaModeliradd_pooling_layerrcL>[TU]U5 Xl[XS9Ulg)z add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer use_mask_token (`bool`, *optional*, defaults to `False`): Whether to use a mask token for masked image modeling. )rN)rrrrr))r$rrvrr%s r&rIJepaModel.__init__js#   )&Pr()rr))FF) rRrSrTrUrrVrrZr[r\s@r&rtrtis( Q{ Qt Q]a Q Qr(rta IJepa Model transformer with an image classification head on top (a linear layer on top of the final hidden states) e.g. for ImageNet. Note that it's possible to fine-tune IJepa on higher resolution images than the ones it has been trained on, by setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained position embeddings to the higher resolution. ) custom_introc ^\rSrSrS\4U4SjjrS S\RS-S\RS-S\S-S\ \ S \ 4 S jjr S r U=r$) IJepaForImageClassificationvrcb>[TU]U5 [USS9UlUR 5 g)NF)rv)rrrtijepa post_init)r$rr%s r&r$IJepaForImageClassification.__init__s(  %@  r(NrClabelsrAkwargsrc UR"U4SU0UD6nURnURURSS95nSnUbUR"X'UR 40UD6n[ UUURURS9$)ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rAr)r=N)losslogits hidden_states attentions) r~last_hidden_state classifierrb loss_functionrrrr) r$rCrrAroutputssequence_outputrrs r&rP#IJepaForImageClassification.forwards/3jj / %=/ /  "33!5!5!!5!<=  %%fdkkLVLD$!//))   r()r~)NNN)rRrSrTrUrrr rWrVrr rrPrZr[r\s@r&r{r{vsp{-1&*04  llT)  t# #'+  +,     r(r{)r^rtr{)r torch.nnr-transformers.models.ijepa.configuration_ijeparrrgmodeling_outputsrrprocessing_utilsrutilsr r r vit.modeling_vitr rrrrr^rtr{__all__rqr(r&rs E&Q&BBeeGmGT/-//" Q%x Q  % "68Q%  % P r(