Zjt SrSSKrSSKrSSKrSSKJr SSKJr SSK J r SSK J r J r SSKJr SS KJrJr SS KJr SS KJrJr SS KJr S SKJr \R8"\5r"SS\R>5r \RBRDS5r#Sr$"SS\R>5r%S1S\RLS\'S\(S\RL4Sjjr)"SS\R>5r*"SS\R>5r+"SS\R>5r,"S S!\R>5r-S"r.S#r/"S$S%\5r0"S&S'\R>5r1\"S(S)\55r2\"S*S+\255r3\"S,S-9"S.S/\ \255r4/S0Qr5g)2zPyTorch ViTDet backbone.N)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutputBaseModelOutput)PreTrainedModel)auto_docstringlogging)can_return_tuple) VitDetConfigcl^\rSrSrSrU4SjrSrS\RS\R4Sjr Sr U=r $) VitDetEmbeddings$z This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cx>[TU]5 URURp2URUR pT[ U[RR5(aUOX"4n[ U[RR5(aUOX34nUSUS-USUS--nX l X0lX@lX`l UR(a@US-n[R"[R "SXqR 55UlOSUl[R$"XEX3S9Ulg)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr"rrrr# num_positions __class__s {/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/vitdet/modeling_vitdet.pyrVitDetEmbeddings.__init__*s !'!;!;V=N=NJ$*$7$79K9Kk#-j+//:R:R#S#SZZdYq #-j+//:R:R#S#SZZdYq !!} 1 5*Q-:VW=:XY $$(&  2 2'!OM')||EKK=RdRd4e'fD $'+D $))L:icU(a USS2SS24nURSn[[R"U55nXf-U:wa [ S5e[ R R5(d Xc:wdXd:waX[RRURSXfS5RSSSS5X44SS S 9nURSSSS5$URSX4S5$) a? Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rrbicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr&jit is_tracingr functional interpolatereshapepermute)r+abs_pos_embeddings has_cls_tokenheightwidth num_positionr6new_abs_pos_embeddingss r/get_absolute_positions'VitDetEmbeddings.get_absolute_positions@s$ !3AqrE!: )//2 499\*+ ;, &TU U 99   ! !dn %']]%>%>"**1d"=EEaAqQ_# &?& "*11!Q1= =%--aC Cr1 pixel_valuesreturnczURSnX R:wa[SURSUS35eURU5nURbbUR SSSS5nX0R URSURSURS5-nUR SSSS5nU$) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr4rT)r9rr=r*r(rCrJ)r+rLr embeddingss r/forwardVitDetEmbeddings.forwardfs#))!, ,, ,!../yaI __\2  # # /#++Aq!Q7J#&A&A(($ 0@0@0CZEUEUVWEX'J$++Aq!Q7Jr1)r"rr#rr(r*) __name__ __module__ __qualname____firstlineno____doc__rrJr&TensorrQ__static_attributes__ __classcell__r.s@r/rr$s5 j,$DLELLU\\r1rcH[S[X5-S- 5nURSU:waq[RR UR SURSS5RSSS5USS9nUR SU5RSS5nOUn[R"U5SS2S4[X- S5-n[R"U5SSS24[X- S5-nXV- US- [X- S5--nXGR5$) aq Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r4rrr3linear)r6r7Ng?) r:maxr9rr@rArBrCr&arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r/ get_rel_posri|s q3v..23L}}Q<'--33 OOAw}}Q/ 4 < bhwkzbhwc,wkc->bhwkN)rir9rBr&einsumview)attnqueries rel_pos_h rel_pos_wrarbqueries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r/!add_decomposed_relative_positionsr|s,%+!N$K!.yIO IFN J3 //*m ICll#3SJOll#3SIO *m*U !Q1d* + , !Q4* + , d: 5{7OP  Kr1c:^\rSrSrSrSU4SjjrSSjrSrU=r$)VitDetAttentionz=Multi-head Attention block with relative position embeddings.c(>[TU]5 URnURnX@lX4-nUS-Ul[ R"X3S-URS9Ul [ R"X35Ul URUl UR(as[ R"[R"SUS-S- U55Ul[ R"[R"SUS-S- U55Ulgg)z Args: config (`VitDetConfig`): Model configuration. input_size (`tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. grbiasr4rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr%r&r'rorp)r+r, input_sizeryrhead_dimr.s r/rVitDetAttention.__init__s   .. "#t^ 99S'@IIc' 060W0W-  0 0\\%++a*Q-6G!6KX*VWDN\\%++a*Q-6G!6KX*VWDN 1r1c URup4pVURU5RX4U-SURS5R SSSSS5nURSX0R-XE-S5R S5upn XR -U RSS5-n UR(a%[XURURXE4XE45n U RSS9n X-nURX0RXES5nUR SSSSS5nURX4US5nURU5nU(a<U RX0RU RSU RS5n X4n U $U4n U $) Nrr3r4rr)ry)r9rrBrrCunbindr transposerr|rorpsoftmaxrlr)r+ hidden_stateoutput_attentionsrwrFrGrxrrnkeysvaluesattention_scoresattention_probsoutputss r/rQVitDetAttention.forwards'3'9'9$ Ehh|$,,Z%DNN\^_gghiklnoqrtuv # AzNN/JFN\^ _ f fgh iv#jj0DNN2r4JJ  0 0@ 4>>4>>F?]c\k  +22r2:&/ #((^^VTVW #++Aq!Q: #++JrJ yy. -55NNO,A,A",EG\G\]_G`O$5G$oGr1)rrrrorprrNF rSrTrUrVrWrrQrYrZr[s@r/r~r~sGX4r1r~input drop_probtrainingrMcUS:XdU(dU$SU- nURS4SURS- --nU[R"X@RUR S9-nUR 5 URU5U-nU$)z[ Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). rr)r)dtypedevice)r9ndimr&randrrfloor_div)rrr keep_probr9 random_tensoroutputs r/ drop_pathr s Cx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr1c^\rSrSrSrS S\S-SS4U4SjjjrS\RS\R4Sjr S\ 4S jr S r U=r $) VitDetDropPathizXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).NrrMc.>[TU]5 Xlgr)rrr)r+rr.s r/rVitDetDropPath.__init__s "r1 hidden_statescB[XRUR5$r)rrr)r+rs r/rQVitDetDropPath.forward s FFr1c SUR3$)Nzp=rr+s r/ extra_reprVitDetDropPath.extra_repr#sDNN#$$r1rr)rSrTrUrVrWfloatrr&rXrQstrrrYrZr[s@r/rrsQb#%$,#$##GU\\GellG%C%%r1rc6^\rSrSrSrSU4SjjrSrSrU=r$)VitDetLayerNormi'a< A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 c>[TU]5 [R"[R "U55Ul[R"[R"U55UlX l U4Ul gr) rrrr%r&onesweightr'repsnormalized_shape)r+rrr.s r/rVitDetLayerNorm.__init__.sR ll5::.>#?@ LL-=!>? !1 3r1cURSSS9nX- RS5RSSS9nX- [R"X0R-5- nUR SS2SS4U-UR SS2SS4-nU$)NrT)keepdimr4)meanpowr&r<rrr)r+xuss r/rQVitDetLayerNorm.forward5s FF1dF # UKKN  4  0 UejjXX. . KK4 & *TYYq$}-E Er1)rrrr)gư>rr[s@r/rr's 4r1rc2^\rSrSrSrU4SjrSrSrU=r$)VitDetResBottleneckBlocki=z The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. c>[TU]5 [R"X$SSS9Ul[ U5Ul[URUl [R"XDSSSS9Ul [ U5Ul [URUl [R"XCSSS9Ul [ U5Ulg)a" Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFrr)paddingrN)rrrr)conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r+r, in_channels out_channelsbottleneck_channelsr.s r/r!VitDetResBottleneckBlock.__init__Cs YY{O $%89 6,,- YY2TU\ab $%89 6,,- YY2!%P $\2 r1cPUnUR5H nU"U5nM X-nU$r)children)r+routlayers r/rQ VitDetResBottleneckBlock.forward[s.]]_E*C%g r1)rrrrrrrrrr[s@r/rr=s 30r1rcr^\rSrSrS\S\SS4U4SjjrS\RS\R4SjrS r U=r $) VitDetMlpid in_featureshidden_featuresrMNc>[TU]5 [R"X#5Ul[ UR Ul[R"X25Ul[R"UR5Ul gr) rrrrfc1rractfc2Dropout dropout_probdrop)r+r,rrr.s r/rVitDetMlp.__init__esV 99[:&++,99_:JJv223 r1rcURU5nURU5nURU5nURU5nURU5nU$r)rrrr)r+rs r/rQVitDetMlp.forwardlsH HHQK HHQK IIaL HHQK IIaLr1)rrrr) rSrTrUrVr:rr&rXrQrYrZr[s@r/rrds=4C4#4$4%,,r1rc HURup#pEXU-- U-nXU-- U-n[RRUSSSUSU45nX6-XG-pUR X(U-XU-X5nUR SSSSSS5R 5R SXU5n XU 44$)a Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrrr4rr3)r9rr@padrlrC contiguous) r window_sizerwrFrGr pad_height pad_width padded_height padded_widthwindowss r/window_partitionrvs/;.@.@+J 44 CJ{22kAI==$$\Aq!Y:3VWL"("5u7H<$$[0+{?Z\gL""1aAq!4??AFFr;eqrG L1 11r1c(UupEUupgURSXE-U-U--nURXU-XQ-XS5n U RSSSSSS5R5n U RXUS5n U SS2SU2SU2SS24R5n U $) a Window unpartition into original sequences and removing padding. Args: windows (`torch.Tensor`): Input tokens with [batch_size * num_windows, window_size, window_size, num_channels]. window_size (`int`): Window size. pad_height_width (`tuple[int]`): Padded height and width (padded_height, padded_width). height_width (`tuple[int]`): Original height and width before padding. Returns: hidden_state: unpartitioned sequences with [batch_size, height, width, num_channels]. rr3rrr4rrN)r9rlrCr) rrpad_height_width height_widthrrrFrGrwrs r/window_unpartitionrs"#3M MFq!m&Bk&QU`&`aJ<<[0,2M{ikL ''1aAq9DDFL$$Z bQL 7F7FUFA 56AACL r1c ^\rSrSrSrSS\S\S\S\SS4 U4S jjjr SS \ RS \S\ \ R\ R4\ \ R-4S jjr S rU=r$) VitDetLayerizCThis corresponds to the Block class in the original implementation.r,drop_path_rateruse_residual_blockrMNc>[T U]5 URnURn[ U[ [ 45(aUOXf4nURn[ U[ [ 45(aUOXw4nUSUS-USUS-4n[R"XQRS9Ul [XS:XaUOX34S9Ul US:a [U5O[R"5Ul[R"XQRS9Ul[%X['XQR(-5S9UlX0lX@lUR.(a[1UUUUS-S9Ulgg) Nrr)r)rr)r,rrr4)r,rrr)rrrr"rlisttuplerr LayerNormlayer_norm_epsrr~ attentionrIdentityrrrr: mlp_ratiomlprrrresidual) r+r,rrrryr"rrr.s r/rVitDetLayer.__init__sF   && #-j4-#H#HZzNf && #-j4-#H#HZzNf  mz!}4jmzRS}6TU \\#+@+@A ( A-=zKC] }N^N^._ +M!% /"0" /q1 (,   a .}>N>NP`cikpbqrM!>>-#@@ %txx =@Y7Z([[ %--aAq9  " " MM-8M "W,r1)rrr rrr rr)rrFr)rSrTrUrVrWrrr:boolrr&rXrrQrYrZr[s@r/rrsMqv!"!49!LO!im! !!L#('||' ' u||U\\) *U5<<-@ @ ''r1rc v^\rSrSrS\SS4U4SjjrS S\RS\S\S \S\ \ -4 S jjr S r U=r $) VitDetEncoderir,rMNc >[TU]5 XlURn[R "SUR USS9Vs/sHo3R5PM nn/n[U5HGnUR[UXFXaR;a UROSXaR;S95 MI [R"U5UlSUlgs snf)Nrcpu)r)rrrF)rrr,num_hidden_layersr&linspaceritemrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r+r,depthrrlayersir.s r/rVitDetEncoder.__init__s  ((-2NN1f>S>SUZch,ij,iq&&(,ijuA MM#1#467;V;V6V 2 2\]'(,I,I'I  ]]6* &+#ksCrroutput_hidden_states return_dictc"U(aSOSnU(aSOSn[UR5H/upxU(aXQ4-nU"X5n U SnU(dM'XiS4-nM1 U(aXQ4-nU(d[SXU455$[UUUS9$)Nrrc3.# UH ocMUv M g7frr').0vs r/ (VitDetEncoder.forward..5sm$[q$[s last_hidden_stater attentions) enumeraterrr ) r+rrr$r%all_hidden_statesall_self_attentionsr" layer_module layer_outputss r/rQVitDetEncoder.forwards#7BD$5b4(4OA#$58H$H!(JM)!,M  &91=M[TU]U5 Xl[U5Ul[ U5UlUR5 gr)rrr,rrPrencoder post_init)r+r,r.s r/rVitDetModel.__init__ds9   *62$V,  r1rMc.URR$rrPr*rs r/get_input_embeddings VitDetModel.get_input_embeddingsn)))r1NrLrr$r%c nUbUOURRnUbUOURRnUbUOURRnUc [ S5eUR U5nUR UUUUS9nUSnU(d U4USS-$[UURURS9$)aw Examples: ```python >>> from transformers import VitDetConfig, VitDetModel >>> import torch >>> config = VitDetConfig() >>> model = VitDetModel(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 768, 14, 14] ```Nz You have to specify pixel_values)rr$r%rrr-) r,rr$r%r=rPrSr rr/) r+rLrr$r%kwargsembedding_outputencoder_outputssequence_outputs r/rQVitDetModel.forwardqs82C1N-TXT_T_TqTq$8$D $++JjJj &1%[TU]U5 [U5Ul[ U5Ul[ URS-5Vs/sHo!RPM snUl UR5 gs snf)Nr) rrrrPrrSrrr num_featuresrT)r+r,rxr.s r/rVitDetBackbone.__init__sh  *62$V, 9>v?W?WZ[?[9\]9\A//9\] ^s A9rMc.URR$rrWrs r/rX#VitDetBackbone.get_input_embeddingsrZr1NrLr$rr%c UbUOURRnUbUOURRnUbUOURRnUR U5nUR USUUS9nU(a UR OUSnSn [URU5HupXR;dMX4- n M U(dU(a U 4USS-n U $U 4USS-n U $[U U(a UR OSURS9$)ap Examples: ```python >>> from transformers import VitDetConfig, VitDetBackbone >>> import torch >>> config = VitDetConfig() >>> model = VitDetBackbone(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```NT)r$rr%rr'r4) feature_mapsrr/) r,r%r$rrPrSrzip stage_names out_featuresr r/) r+rLr$rr%r\r]rrrjstagerrs r/rQVitDetBackbone.forwards+<&1%script_if_tracingrir|r~rXrrrrrrrrrrrr7rQrc__all__r'r1r/rs &!H9?-,-.   H %UryyUp!3!3H&R;bii;~ U\\ e T V[VbVb  %RYY %bii,$ryy$N $2@>M,M`5 BII5 p!+O!+!+HD 'D D N M ]$9M  M ` Er1