Zj o SrSSKrSSKrSSKrSSKrSSKJrJr SSKJ r SSK J r SSK JrJr SSKJrJrJrJr SS KJr SS KJrJr SS KJr S S KJr \R<"\5r S7S\!\!\"44Sjjr#"SS\RH5r%"SS\RL5r'"SS\RP5r)"SS\RT5r+"SS\RP5r,S8S\R S\-S\"S\R 4Sjjr."SS\RP5r/S9S jr0"S!S"\RP5r1"S#S$\RP5r2"S%S&\RP5r3"S'S(\RP5r4"S)S*\RP5r5\"S+S,\55r6\"S-S.\655r7\"S/S09"S1S2\655r8\"S3S09"S4S5\\655r9/S6Qr:g):z9PyTorch BiT model. Also supports backbone for ViT hybrid.N)Tensornn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)BackboneOutputBaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)can_return_tuple) BitConfigreturnc"SnUcUS- X1S- --S-nX4$[U[5(a`UR5nUS:Xa/US:Xa!X1S- -S-S:XaUS- X1S- --S-nX4$SnSnX4$US:XaSnX4$US- X1S- --S-nX4$)a< Utility function to get the tuple padding value given the kernel_size and padding. Args: padding (Union[`str`, `int`], *optional*): Padding value, can be either `"same"`, `"valid"`. If a different value is provided the default padding from PyTorch is used. kernel_size (`int`, *optional*, defaults to 7): Kernel size of the convolution layers. stride (`int`, *optional*, defaults to 1): Stride value of the convolution layers. dilation (`int`, *optional*, defaults to 1): Dilation value of the convolution layers. FrsamerTvalid) isinstancestrlower)padding kernel_sizestridedilationdynamics u/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/bit/modeling_bit.pyget_padding_valuer")sGQJ(Ao">>1D'3--/ f {!O <AQF"QJ(Ao*FF1L      G   h/&BBqHG  cB^\rSrSrSrSU4SjjrSrSrU=r$)WeightStandardizedConv2dRzConv2d with Weight Standardization. Used for ViT Hybrid model. Paper: [Micro-Batch Training with Batch-Channel Normalization and Weight Standardization](https://huggingface.co/papers/1903.10520) c >[XSXFS9upZ[T U] UUUUUUUUS9 U (a[X4U5UlOSUlXlg)N)rr)rrrgroupsbias)r"super__init__ DynamicPad2dpadeps) self in_channel out_channelsrrrrr(r)r. is_dynamic __class__s r!r+!WeightStandardizedConv2d.__init__Ys]0Vg       #KBDHDHr#c URbURU5n[RRURR SUR S5SSSSURS9RUR5n[RRXURURURURUR5nU$)NrT)trainingmomentumr.)r-r functional batch_normweightreshaper1r. reshape_asconv2dr)rrrr()r/ hidden_stater<s r!forward WeightStandardizedConv2d.forwardvs 88 88L1L)) KK  4#4#4b 94PT_bhlhphp* *T[[ ! }}++ $))T[[$,, W[WbWb r#)r.r-)rSAMErrFgư> __name__ __module__ __qualname____firstlineno____doc__r+rA__static_attributes__ __classcell__r3s@r!r%r%Rs+  :  r#r%c6^\rSrSrSrSU4SjjrSrSrU=r$)BitGroupNormActivationzI A module that combines group normalization with an activation function. c>[TU]URX#US9 U(a[URUlg[ R"5Ulg)N)r.affine)r*r+ num_groupsr hidden_act activationrIdentity)r/config num_channelsr.rQapply_activationr3s r!r+BitGroupNormActivation.__init__s? **L&Q $V%6%67DO kkmDOr#c[RRXRURUR UR 5nURU5nU$N)rr: group_normrRr<r)r.rT)r/r@s r!rABitGroupNormActivation.forwardsF}}// oot{{\`\e\egkgogop |4 r#)rT)gh㈵>TTrDrLs@r!rNrNs,r#rNc6^\rSrSrSrSU4SjjrSrSrU=r$)r,z A module that wraps dynamic padding of any input, given the parameters of the convolutional layer and the input hidden states. c>[TU]5 [U[5(aX4n[U[5(aX"4n[U[5(aX34nXlX lX0lX@lSnXPlg)Ncp[[R"X- 5S- U-US- U--S-U- S5$)Nrr)maxmathceil)xrrrs r!compute_padding.DynamicPad2d.__init__..compute_paddings@ !*-1V;{QRZ>ZZ]^^abbdef fr#) r*r+rintrrrvaluerf)r/rrrrirfr3s r!r+DynamicPad2d.__init__sp  k3 ' '&4K fc " "%F h $ $ +H&    g /r#c UR5SSup#URX RSURSURS5nURX0RSURSURS5nUS:dUS:a=[ R RUUS-XUS-- US-XDS-- /URS9nU$)Nrrr)ri) sizerfrrrrr:r-ri)r/input input_height input_widthpadding_height padding_widths r!rADynamicPad2d.forwards$)JJL$5! --l[U[RR5(aUOX4n[U[RR5(aUOX"4n[U[RR5(aUOX34n[TU]XXSU5 U(a[ XX65Ulg[R"5Ulgr[) r collectionsabcIterabler*r+r,r-rrU) r/rrr ceil_moder padding_valueuse_dynamic_paddingr3s r!r+BitMaxPool2d.__init__s&0 [__=U=U%V%Vk]h\v %fkoo.F.FGGfM])(KOO4L4LMM8T\Sg gK #KQDH{{}DHr#cURU5n[RRXRUR UR URUR5$r[) r-rr: max_pool2drrrrr{r/ hidden_statess r!rABitMaxPool2d.forwardsK/ }}'' ++T[[$,, W[WeWe  r#)r-)NrF)rrrT) rErFrGrHrhr+rArJrKrLs@r!rurus3 %%%&  r#rucF^\rSrSrSrS\4U4SjjrS\S\4SjrSr U=r $) BitEmbeddingszD BiT Embeddings (stem) composed of a single aggressive convolution. rVc  >[TU]5 [URURSSSUR S9Ul[SSURS9Ul UR b9UR R5S:Xa[R"5Ul O[R"SS S 9Ul URS :wa[!XRS 9UlO[R"5UlURUlg) Nr:0yE>)rrr.rr)rrr}rC)rrrrr7)rri preactivationrW)r*r+r%rWembedding_sizeglobal_padding convolutionruembedding_dynamic_paddingpoolerupperrrUr- ConstantPad2d layer_typerNnormr/rVr3s r!r+BitEmbeddings.__init__s 3     ! !))  #qPVPpPpq   ,1F1F1L1L1NRX1X{{}DH'' CHDH    /.vDYDYZDI DI"//r# pixel_valuesrcURSnX R:wa [S5eURU5nUR U5nUR U5nUR U5nU$)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)shaperW ValueErrorrr-rr)r/rrW embeddings r!rABitEmbeddings.forwardsp#))!, ,, ,w $$\2 HHY' IIi( KK * r#)rrrWr-r) rErFrGrHrIrr+rrArJrKrLs@r!rrs,0y06Fvr#rrn drop_probr8cUS:XdU(dU$SU- nURS4SURS- --nU[R"X@RUR S9-nUR 5 URU5U-nU$)z[ Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). r7rr)r)dtypedevice)rndimtorchrandrrfloor_div)rnrr8 keep_probr random_tensoroutputs r! drop_pathrs Cx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr#c^\rSrSrSrS S\S-SS4U4SjjjrS\RS\R4Sjr S\ 4S jr S r U=r $) BitDropPathi%zXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).Nrrc.>[TU]5 Xlgr[)r*r+r)r/rr3s r!r+BitDropPath.__init__(s "r#rcB[XRUR5$r[)rrr8rs r!rABitDropPath.forward,s FFr#c SUR3$)Nzp=r)r/s r! extra_reprBitDropPath.extra_repr/sDNN#$$r#rr[)rErFrGrHrIfloatr+rrrArrrJrKrLs@r!rr%sQb#%$,#$##GU\\GellG%C%%r#rcdUn[U[XS- -5U-U-5nUSU-:aX1- nU$)Nrg?)rbrh)ridivisor min_value new_values r!make_divr3sCIIs5Q;#677BWLMI3; r#cF^\rSrSrSrSU4SjjrSrSrU=r$)BitPreActivationBottleneckLayeri;zPre-activation (v2) bottleneck block. Follows the implementation of "Identity Mappings in Deep Residual Networks": https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua Except it puts the stride on 3x3 conv when available. c  >[T U]5 U=(d UnU=(d Un[X4-5n U (a[UUUUSS9UlOSUl[ X5Ul[X+SSURS9Ul [ XS9Ul [XSXXSURS9Ul [ X5Ul [XSSURS9Ul U S :a[U 5Ulg[R "5Ulg) NTrpreactrrr.rrr)rr(r.rr)r*r+rBitDownsampleConv downsamplerNnorm1r%rconv1norm2conv2norm3conv3rrrUr) r/rV in_channelsr1 bottle_ratiorrfirst_dilationr(drop_path_rateis_first_layer mid_channelsr3s r!r+(BitPreActivationBottleneckLayer.__init__Cs '38#2{  ;< / DO#DO+F@ -kPT^d^s^st +FN - &T[a[p[p  ,FA -l!QU_e_t_tu 8F8J^4PRP[P[P]r#c0URU5nUnURbURU5nURU5nURUR U55nUR UR U55nURU5nX-$r[)rrrrrrrr)r/rhidden_states_preactshortcuts r!rA'BitPreActivationBottleneckLayer.forwardos#zz-8! ?? &';[T U]5 U=(d UnU=(d Un[X4-5n U (a[UUUUSS9UlOSUl[ X+SSUR S9Ul[XS9Ul [ U U SUUUSUR S9Ul [XS9Ul [ XSSUR S9Ul [XSS 9Ul U S :a [U 5O[R "5Ul[$UR&Ulg) NFrrrrrr)rrr(r.rrWrXr)r*r+rrrr%rrrNrrrrrrrrUrrrSrT) r/rVrr1rrrrr(rrmid_chsr3s r!r+BitBottleneckLayer.__init__s '38#2{ <67 / DO#DO-kA4Y_YnYno +FI -   #))   ,FI -gQDZ`ZoZop +F`ef 8F8J^4PRP[P[P] !2!23r#cZUnURbURU5nURU5nURU5nURU5nUR U5nUR U5nUR U5nURU5nURX-5nU$r[) rrrrrrrrrT)r/rrs r!rABitBottleneckLayer.forwards  ?? &}5H =1  =1  =1  =1  =1  =1 }5  (@A r#) rTrrrrrrrrrrDrLs@r!rrs0f /4br#rc6^\rSrSrSU4SjjrSrSrU=r$)ric >[TU]5 [X#SUSURS9UlU(a[ R "5Ulg[XSS9Ulg)Nrr)rr.rFr) r*r+r%rconvrrUrNr)r/rVrr1rrr3s r!r+BitDownsampleConv.__init__sX , qT6K`K`   KKM (\ab r#cBURURU55$r[)rr)r/res r!rABitDownsampleConv.forwardsyy1&&r#)rr)rT)rErFrGrHr+rArJrKrLs@r!rrs   $''r#rcL^\rSrSrSrS U4SjjrSrS\S\4SjrSr U=r $) BitStageiz/ A ResNet v2 stage composed by stacked layers. c `>[TU]5 US;aSOSn URS:Xa[n O[n Un [ R "5Ul[U5HMn URXU5upMnURR[U 5U "UU UUUUU U US9 5 Un Un MO g)N)rrrr bottleneck)rrrrrr) r*r+rrrr Sequentiallayersrange_get_updated_hyperparameters add_moduler)r/rVrr1rrdepthr layer_dropoutr layer_clsprev_chs layer_idxrrr3s r!r+BitStage.__init__s &&0a    ,*I7Immo uI595V5V=6 2FN KK " "I !%!-#1#1#1  $H%N+&r#c@U(aX1nOSnUS:waSnUS:HnX$U4$)zd Get the new hyper-parameters with respect to the previous ones and the index of the current layer. r7rr)r/rrrrrs r!r%BitStage._get_updated_hyperparameterss4 *5N N >F"a~55r#rnrcVUn[UR5H up4U"U5nM U$r[) enumerater)r/rnr@_layers r!rABitStage.forward"s, !$++.HA .L/r#)r)rN) rErFrGrHrIr+rrrArJrKrLs@r!rrs3,&\6 Vr#rc V^\rSrSrS\4U4SjjrSrS S\S\S\S\ 4S jjr S r U=r $) BitEncoderi)rVc >[TU]5 [R"/5UlUR nSnSn[ R"[R"SUR[UR555RUR5Vs/sHnUR5PM nn[[!URUR"U55HYunupn UR%XsXU5upn['UUU U UUU S9n U nX<-nURR)[+U5U 5 M[ gs snf)Nrr)rrrr)r*r+r ModuleListstagesrrrnplinspacersumdepthssplittolistrzip hidden_sizesrrrr)r/rVrcurrent_striderrelayer_dropouts stage_idx current_depthcurrent_hidden_sizerr1rstager3s r!r+BitEncoder.__init__*s5 mmB' ((\\"++a1F1FFMMHZ"[\bbcicpcpq q HHJq  OX  v22N CO JIJ M.2-N-N+>&. *L(!#+E$H  $N KK " "3y>5 9+O  sEcv[X5R-5nUS:XaSOSnX%R:aXG-nSnXgU4$)Nrrr)r width_factor output_stride)r/r rr rrVr1rs r!r'BitEncoder._get_updated_hyperparametersPsG 36I6I IJ 1n! 11 1  HFX--r#r@output_hidden_states return_dictrcU(aSOSnURHnU(aXA4-nU"U5nM U(aXA4-nU(d[SX455$[UUS9$)Nrc3.# UH ocMUv M g7fr[r).0vs r! %BitEncoder.forward..gsS$Aq$As )last_hidden_stater)rtupler )r/r@rrr stage_modules r!rABitEncoder.forwardXsk3  KKL# - ? ' 5L ( )O;MS\$ASS S-*'  r#)r)FT) rErFrGrHrr+rrboolr rArJrKrLs@r!rr)sF$:y$:L.]a " :> UY '  r#rc`\rSrSr%\\S'SrSrSrS/r \ R"5S5r Sr g ) BitPreTrainedModeliorVbit)imagerrc[U[R5(a![R"UR SSS9 g[U[R 5(a[R"UR [R"S5S9 URbz[RRRUR 5up#US:aS[R"U5- OSn[R"URU*U5 gg[U[R[R45(a[R "UR S5 [R "URS5 [#USS5ba[R$"UR&5 [R("UR*5 [R$"UR,5 ggg) Nfan_outrelu)mode nonlinearity)arr running_mean)rrConv2dinitkaiming_normal_r<Linearkaiming_uniform_rcsqrtr)r_calculate_fan_in_and_fan_outuniform_ BatchNorm2d GroupNorm constant_getattrzeros_r+ones_ running_varnum_batches_tracked)r/modulefan_inrbounds r! _init_weights BitPreTrainedModel._init_weightswsE fbii ( (  YV T  * *  ! !&--499Q< @{{&!HHMMGG V 17!DIIf-- fkkE659' > ? ? NN6==! , NN6;; *v~t4@ F//0 6--. F667A@r#rN)rErFrGrHr__annotations__base_model_prefixinput_modalitiesmain_input_name_no_split_modulesrno_gradr?rJrr#r!r!r!os: !$O() ]]_88r#r!c `^\rSrSrU4Sjr\S S\S\S-S\S-S\4Sjj5r S r U=r $) BitModelicF>[TU]U5 Xl[U5Ul[ U5UlURS:Xa[XRSS9O[R"5Ul [R"S5UlUR5 g)Nrr6r)rr)r*r+rVrembedderrencoderrrNrrrUrAdaptiveAvgPool2dr post_initrs r!r+BitModel.__init__s   %f- !&)   O3 #68K8KB8O P **62  r#Nrrrrc HUbUOURRnUbUOURRnURU5nUR XRUS9nUSnUR U5nUR U5nU(d Xx4USS-$[UUURS9$)Nrrrr)r pooler_outputr) rVrrrJrKrrr r) r/rrrkwargsembedding_outputencoder_outputsr pooled_outputs r!rABitModel.forwards%9$D $++JjJj &1%[TU]U5 URUl[U5Ul[ R "[ R"5URS:a.[ R"URSUR5O[ R"55Ul UR5 g)Nrr6) r*r+ num_labelsrHr"rrFlattenr/rrU classifierrMrs r!r+"BitForImageClassification.__init__s   ++F#-- JJLEKEVEVYZEZBIIf))"-v/@/@ A`b`k`k`m  r#Nrlabelsrrrc JUbUOURRnURXUS9nU(a UROUSnUR U5nSn UbUR X(UR5n U(dU4USS-n U bU 4U -$U $[ XURS9$)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NrPrr)losslogitsr)rVrr"rQr^ loss_functionr r) r/rr`rrrRoutputsrUrcrbrs r!rA!BitForImageClassification.forwards&1%[TU]U5 [U5UlUR/UR -UlUR5 gr[)r*r+rHr"rr num_featuresrMrs r!r+BitBackbone.__init__sD  F##223f6I6II r#Nrrrrc UbUOURRnUbUOURRnURUSSS9nURnSn[ UR 5HupXR;dMXvU4- nM U(dU4n U(aXR4- n U $[UU(aURSS9$SSS9$)a' Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> processor = AutoImageProcessor.from_pretrained("google/bit-50") >>> model = AutoBackbone.from_pretrained("google/bit-50") >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) ```NTrPr) feature_mapsr attentions) rVrrr"rr stage_names out_featuresr ) r/rrrrRrerroidxr rs r!rABitBackbone.forwards<&1%rs@  &!H .,-(   H %&ERWY]R]L^&R-ryy-`R\\$02990f 2<< 6/BII/f U\\ e T V[VbVb  %")) %A(biiA(HFFR' '.GryyGTC C L88862 !2 2 j ,s 2,s  ,s^ B -!3B  B J Yr#