Zj4B SrSSKrSSKrSSKrSSKJr SSKJr SSKJ r \R"\ 5r "SS\R5r"S S 5rS r"S S 5rSS\S-S\S-S\S-S\S-4SjjrSrg)zACollection of utils to be used by backbones and their components.N) repo_exists)logging)maybe_install_capturing_hooksc\rSrSrSrSrSrg) BackboneTypetimm transformersN)__name__ __module__ __qualname____firstlineno__TIMM TRANSFORMERS__static_attributes__r l/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/backbone_utils.pyrrs D!Lrrc^\rSrSrSrS\S-S\S-4SjrSr\S5r \ RS\\ 4S j5r \S 5r \ RS\ \S 4\\-4S j5r U4S jrSrU=r$)BackboneConfigMixin#zn A Mixin to support handling the `out_features` and `out_indices` attributes for the backbone configurations. out_featuresN out_indicescXl[U[5(a [U5OUUlUR 5 URURp!Uc-Uc*[ UR5S- /nURS/nOUUc.Ub+UVs/sHo0RRU5PM nnO$Uc!UbUVs/sHo@RUPM nnXsUlUlUR 5 gs snfs snf)a( Sets output indices and features to new values and aligns them with the given `stage_names`. If one of the inputs is not given, find the corresponding `out_features` or `out_indices` for the given `stage_names`. Args: out_features (`list[str]`, *optional*): The names of the features for the backbone to output. Defaults to `config._out_features` if not provided. out_indices (`list[int]` or `tuple[int]`, *optional*): The indices of the features for the backbone to output. Defaults to `config._out_indices` if not provided. Nr) _out_features isinstancetuplelist _out_indicesverify_out_features_out_indiceslen stage_namesindex)selfrrlayeridxs r"set_output_features_output_indices6BackboneConfigMixin.set_output_features_output_indices(s *1;K1O1OD-U` ,,.%)$6$68I8Ik  <#7t//0145K ,,R01L  \%=FRSlU++11%8lKSK  !k&==HI[c,,S1[LI1=-D- ,,. TIs $C9C>c^TRc [S5eTRGb0[TR[45(d![S[ TR535e[ U4SjTR55(a%[STRSTR35e[TR5[[TR55:wa[STR35eTRTRVs/sHoTR;dMUPM sn=n:wa[SUSTR35eTRGbY[TR[5(d![S [ TR535e[U4S jTR55n[ U4S jU55(a%[S TRS TR35e[U5[[U55:wa4STR3nXCTR:waSUS3OS- n[U5eU[[U55:waN[[UTR5SS9VVs/sHupVUPM nnn[SUSTR35eTRbTRb{[TR5[TR5:wa [S5eTRTRVs/sHnTRUPM sn:wa [S5egggs snfs snnfs snf)zO Verify that out_indices and out_features are valid for the given stage_names. Nz2Stage_names must be set for transformers backbonesz out_features must be a list got c3@># UHoTR;v M g7fN)r$).0featr&s r FBackboneConfigMixin.verify_out_features_out_indices..VsO# UH$oS:aU[TR5-OUv M& g7f)rN)r#r$r.r(r&s rr0r1gs.$r`qY\!GS3t/?/?+@%@QT%T`qs,/c3n># UH*o[[TR55;dM&Uv M, g7fr-)ranger#r$r3s rr0r1hs*^"23s4K[K[G\A]6]33"2s%5 5z2out_indices must be valid indices for stage_names z, got z1out_indices must not contain any duplicates, got z(equivalent to z))c US$)Nrr )xs rEBackboneConfigMixin.verify_out_features_out_indices..rs efgheir)keyz?out_indices must be in the same order as stage_names, expected zHout_features and out_indices should have the same length if both are setzQout_features and out_indices should correspond to the same stages if both are set) r$ ValueErrorrrr typeanyr#setr!rsortedzip)r&r/ sorted_featspositive_indicesmsg_r(sorted_negatives` rr"3BackboneConfigMixin.verify_out_features_out_indicesLsG    #QR R    )d004':: #CDI[I[D\C]!^__OD[TU]5nURSS5US'URSS5US'U$)z Serializes this instance to a Python dictionary. Override the default `to_dict()` from `PreTrainedConfig` to include the `out_features` and `out_indices` attributes. rNrr!r)superto_dictpop)r&output __class__s rrVBackboneConfigMixin.to_dicts@ "!'OT!B~ & >4 @} r)rr!)r rrr__doc__r r)r"propertyrsetterstrrrintrVr __classcell__rYs@rrr#s"/Tk"/D["/H0vd""]c]] !!\uS#Xc'B\\rrcF^[R"T5U4Sj5nU$)aW Wrapper to filer out `hidden_states` as backbones tend to always use them to get their feature maps, i.e. they also always output `hidden_states`. This controls for user-defined behavior again. NOTE: We assume a `can_return_tuple` decorator to be applied before so that we always expect a dict like object to remove the hidden states. c>URS[URSS55nT"U/UQ70UD6nU(d=UR5VVs0sHupVUS;dM XV_M nnn[ U5"S0UD6nU$s snnf)Noutput_hidden_statesF hidden_statesr )getgetattrconfigitemsr=) r&argskwargsrdrXkvfiltered_output_dataforward_functions rwrapper,filter_output_hidden_states..wrappers%zz*@'$++WmotBuv!$888#5;\\^#b^TQqQ`GaDAD^ #b&\9$89F $cs  A;A;) functoolswraps)rorps` rfilter_output_hidden_statesrts(__%&' NrcX^\rSrSr%Sr\S-\S'Sr\\S'SU4Sjjr U4Sjr SSjr SS jr \ S 5r\RS \\4S j5r\ S 5r\RS\\\\-4Sj5r\ S5r\ S5rSrSS\S-S\S-S\S-4SjjrSrU=r$) BackboneMixinN backbone_typeThas_attentionsc>[TU]"U0UD6 URSS5nUb[RUlO[R UlUR [R:XaURUS9 gUR [R :XaUR5 g[SUR S35e)z Method to initialize the backbone. This method is called by the constructor of the base class after the pretrained model weights have been loaded. timm_backboneN)backbonezbackbone_type z not supported.) rU__init__rWrrrxr_init_timm_backbone_init_transformers_backboner<)r&rjrkr{rYs rr}BackboneMixin.__init__s $)&) ?D9  $!-!2!2D !-!:!:D    !2!2 2  $ $m $ <   <#<#< <  , , .~d.@.@-AQR Rrc8>[TU]5 [U5 g)am Override `post_init` to always install capturing hooks, as backbone will ALWAYS capture outputs. We need to do it in `post_init`, as modules need to be already instantiated. It avoids some mixups with `torch.compile`, as the first hook installation will need/create a graph break, which can clash with external user call such as `model = torch.compile(model...)`. N)rU post_initr)r&rYs rrBackboneMixin.post_inits %d+rcz[URSS5n[URSS5nURRVs/sHoDSPM snUlURRVs/sHoDSPM snUl[ URR5nURR5nUbX&:wa[SUS35eUbX0R:wa[SUS 35eURURlURRXe5 gs snfs snf) zZ Initialize the backbone model from timm. The backbone must already be loaded to backbone rNr$modulenum_chsz!Config has `out_features` set to z which doesn't match `out_features` from backbone's feature_info. Please check if your checkpoint has correct out features/indices saved.z Config has `stage_names` set to z which doesn't match `stage_names` from backbone's feature_info. Please check if your checkpoint has correct `stage_names` saved.) rgrh feature_infoinfor$ num_featuresr r module_namer<r))r&r|out_features_from_configstage_names_from_configstagerrs rr~!BackboneMixin._init_timm_backbones; $+4;;#M ")$++}d"K :B9N9N9S9ST9S(O9ST;C;P;P;U;UV;U%9-;UV800<<= ,,88: # /4L4\34L3MNxx  # .3JN^N^3^23J2KLqq  #'"2"2  66|Q)UVs D35D8c|URRUlURR5 SUlgr-)rhr$r"rrIs rr)BackboneMixin._init_transformers_backbones,;;22 335 rc.URR$r-)rhrrIs rrBackboneMixin.out_featuress{{(((rrc$XRlg)rLN)rhrrNs rrrs $0  rc.URR$r-)rhr!rIs rrBackboneMixin.out_indices s{{'''rrc$XRlg)rRN)rhrrSs rrrs #. rc|[UR5VVs0sHupX RU_M snn$s snnfr-) enumerater$r)r&irs rout_feature_channels"BackboneMixin.out_feature_channelss:=FdFVFVO++6+: O ++?+C Or)NNNN)r[enumrrrhuggingface_hubrutilsrutils.output_capturingr get_loggerr rEnumrrrtrvr^rrrr rrrsH 'A   H %"499" zzz*|]|]B$(&*)-'+ /#Dj/#t/# $; /#  /#dr