Zj8\hSSKJr SSKrSSKJr SSKJr SSKJ r SSK J r SSK J r Jr SSKJrJr SS KJr SS KJrJrJr SS KJr SS KJr S SKJr "SS\R<5r"SS\R<5r "SS\R<5r!S4S\R<S\RDS\RDS\RDS\RDS-S\#S\#S\\4Sjjr$Sr%S\RDS \&S!\RD4S"jr'S#\RDS$\RDS%\RDS&\RDS!\(\RD\RD44 S'jr)"S(S)\R<5r*"S*S+\ 5r+"S,S-\R<5r,\"S.S/\55r-\"S0S19"S2S3\-55r.S/S3/r/g)5)CallableN)initialization)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPooling)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstring torch_int)merge_with_config_defaults)capture_outputs)MLCDVisionConfigcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)MLCDMLP%c>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gN) super__init__configr hidden_act activation_fnnnLinear hidden_sizeintermediate_sizefc1fc2selfr __class__s w/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/mlcd/modeling_mlcd.pyrMLCDMLP.__init__&sb  #F$5$5699V//1I1IJ99V55v7I7IJ hidden_statesreturnclURU5nURU5nURU5nU$r)r"rr#)r%r*s r'forwardMLCDMLP.forward-s4/ **=9 / r))rrr"r#) __name__ __module__ __qualname____firstlineno__rtorchTensorr-__static_attributes__ __classcell__r&s@r'rr%s)KU\\ellr)rc^\rSrSr%\R \S'S S\S\SS4U4Sjjjr S\S \S\R 4S jr S r U=r $) MLCDRotaryEmbedding4inv_freqdimthetar+Nc >[TU]5 XlX lSU[R "SUS[R S9U- -- nURSUSS9 g)N?rdtyper;F persistent)rrr<r=r3arangefloatregister_buffer)r%r<r=r;r&s r'rMLCDRotaryEmbedding.__init__7sU  %ELLC%++$NQT$TUV ZeDr)num_patches_heightnum_patches_widthc~[R"XRRS9R S5R SU5n[R"X RRS9R S5R US5n[R "UR5UR5/SS9n[X5n[R"X`RRURRS9n[R"XpR5nXRS5n U $)aE Calculate the Rotary Position Embedding (RoPE) for MLCDVisionModel based on the grid size. Args: num_patches_height (int): Number of patches in the height dimension. num_patches_width (int): Number of patches in the width dimension. Returns: torch.Tensor: Rotary positional embeddings for the given grid size. )devicerrr<)rLrB) r3rEr;rL unsqueezeexpandstackflattenmaxrBouter) r%rIrJhpos_idswpos_idspos_ids max_grid_sizeseqrotary_pos_emb_fullrotary_pos_embs r'r-MLCDRotaryEmbedding.forward>s LL+MM4H4H I S STU V ] ]^`bs t  LL*==3G3G H R RST U \ \]oqs t  ++x//183C3C3EFBO.B ll=1E1ET]]M`M`a#kk#}}=-5==a@r))r<r=)g@) r/r0r1r2r3r4__annotations__intrFrr-r5r6r7s@r'r9r94sTllECEEDEE##%,,r)r9c^\rSrSrS\4U4SjjrS\RS\S\S\R4Sjr S \RS\R4S jr S r U=r $) MLCDVisionEmbeddings_rcv>[TU]5 XlURUlUR UlUR Ul[R"[R"UR55Ul [R"URURUR UR SS9UlUR UR -S-UlURS-UlUR#S[R$"UR 5R'S5SS9 g)NF) in_channels out_channels kernel_sizestridebiasr@r position_idsrrMrC)rrrr embed_dim image_size patch_sizer Parameterr3randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positionsrGrErPr$s r'rMLCDVisionEmbeddings.__init__`s  ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1 ^U\\$:L:L-M-T-TU\-]jopr) embeddingsheightwidthr+cURSS- nURRRS5nURSS- n[R R 5(d%XF:Xa X#:XaURUR5$USS2SS24nUSS2SS24nURSn X R-n X0R-n [US-5n URSXU 5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU 5n[R "Xx4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNrMg?rr@bicubicF)sizemode align_cornersrN)shapeposition_embeddingweightrOr3jit is_tracingrhrlrreshapepermuter functional interpolateviewcat) r%rvrwrxrsrrtclass_pos_embedpatch_pos_embedr< new_height new_widthsqrt_num_positionss r'interpolate_pos_encoding-MLCDVisionEmbeddings.interpolate_pos_encodingusi!&&q)A- !44;;EEaH*003a7 yy##%%+*F6?**4+<+<= =,QU3,QU3r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr) pixel_valuescHURSnURRRnURUR US95nUR S5R SS5nURRUSS5n[R"XT/SS9nU$)NrrAr@rrMrN) r~rrrrBtorR transposerorPr3r)r%r batch_size target_dtype patch_embeds class_embedsrvs r'r-MLCDVisionEmbeddings.forwards!''* ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY ;C r))rorrjrkrsrtrrrl)r/r0r1r2rrr3r4r^r FloatTensorr-r5r6r7s@r'r`r`_sgq/q*'D5<<'D'DUX'D]b]i]i'DR E$5$5 %,,  r)r`modulequerykeyvalueattention_maskscalingdropoutkwargsc [X R5n[X0R5n [R"XR SS55U-n UbX-n [ R RU S[RS9RUR5n [ R RXURS9n [R"X5n U R SS5R5n X4$)Nr@rrM)r<rB)ptrainingr) repeat_kvnum_key_value_groupsr3matmulrrrsoftmaxfloat32rrBrr contiguous) rrrrrrrr key_states value_states attn_weights attn_outputs r'eager_attention_forwardrs3 ; ;