Zj SSKrSSKrSSKJr SSKJr SSKrSSKrSSK J s J r SSKJ r J r SSKJr SSKJr SSKJrJrJr SS KJr SS KJrJr SS KJr SS KJrJ r J!r! SS K"J#r# SSK$J%r% SSK&J'r' \"5(aSSK(J)r) \!"5(a SSK*J+r+ SSK,J-r- \\ "SS9"SS\555r.SPS\RS\RS\R4Sjjr/S\ S\ S\ 4Sjr0S\RS\RS\R4Sjr1"S S!\ Rd5r3S\ S\ S"\4S\ 4S#jr5S\RS\RS"\4S\R4S$jr6"S%S&\ Rd5r7"S'S(\ Rd5r8"S)S*\ Rd5r9SQS+\ RdS,\RS-\RS.\RS/\RS-S0\:S1\:4S2jjr;"S3S4\ Rd5r<"S5S6\ Rd5r=SRS7\RS8\:S9\>S\R4S:jjr?"S;S<\ Rd5r@"S=S>\ Rd5rA"S?S@\ Rd5rB"SASB\5rC"SCSD\ R5rE"SESF\ Rd5rF"SGSH\ Rd5rG"SISJ\ Rd5rH\ "SKSL\55rI\ "SMS9"SNSO\I55rJSLSO/rKg)SN)Callable) dataclass)Tensornn)initialization)ACT2FN) ModelOutputis_scipy_availablerequires_backends)GradientCheckpointingLayer)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringis_accelerate_available)merge_with_config_defaults)capture_outputs) EomtConfig)linear_sum_assignment) PartialState)reducea Class for outputs of [`EomtForUniversalSegmentationOutput`]. This output can be directly passed to [`~EomtImageProcessor.post_process_semantic_segmentation`] or [`~EomtImageProcessor.post_process_instance_segmentation`] or [`~EomtImageProcessor.post_process_panoptic_segmentation`] to compute final segmentation maps. Please, see [`~EomtImageProcessor] for details regarding usage. ) custom_introcD\rSrSr%SrSr\RS-\S'Sr \RS-\S'Sr \RS-\S'Sr \RS-\S'Sr \ \RS-\S'Sr\ \RS-\S 'Sr\\R"S-\S 'S rg) "EomtForUniversalSegmentationOutput3a loss (`torch.Tensor`, *optional*): The computed loss, returned when labels are present. class_queries_logits (`torch.FloatTensor`): A tensor of shape `(batch_size, num_queries, num_labels + 1)` representing the proposed classes for each query. Note the `+ 1` is needed because we incorporate the null class. masks_queries_logits (`torch.FloatTensor`): A tensor of shape `(batch_size, num_queries, height, width)` representing the proposed masks for each query. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Last hidden states (final feature map) of the last layer. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states all layers of the model. attentions (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `tuple(torch.FloatTensor)` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Self and Cross Attentions weights from transformer decoder. patch_offsets (`list[torch.Tensor]`, *optional*): list of tuples indicating the image index and start and end positions of patches for semantic segmentation. Nlossclass_queries_logitsmasks_queries_logitslast_hidden_state hidden_states attentions patch_offsets)__name__ __module__ __qualname____firstlineno____doc__rtorch FloatTensor__annotations__r r!r"r#tupler$r%listr__static_attributes__r&w/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/eomt/modeling_eomt.pyrr3s*&*D%  d ")59%++d2959%++d2926u((4/659M5**+d2926Je''(4/6/3M4 %,3r2rinput_featurespoint_coordinatesreturnc UR5S:XaSnURS5n[RRR "USU-S- 40UD6nU(aUR S5nU$)a A wrapper around `torch.nn.functional.grid_sample` to support 3D point_coordinates tensors. Args: input_features (`torch.Tensor` of shape (batch_size, channels, height, width)): A tensor that contains features map on a height * width grid point_coordinates (`torch.Tensor` of shape (batch_size, num_points, 2) or (batch_size, grid_height, grid_width,: 2)): A tensor that contains [0, 1] * [0, 1] normalized point coordinates add_dim (`bool`): boolean value to keep track of added dimension Returns: point_features (`torch.Tensor` of shape (batch_size, channels, num_points) or (batch_size, channels, height_grid, width_grid): A tensor that contains features for points in `point_coordinates`. rTg@?)dim unsqueezer,r functional grid_samplesqueeze)r4r5add_dimkwargspoint_featuress r3 sample_pointrB^st(!#-77:XX((44^SK\E\_bEbmflmN'//2 r2inputslabelscUR5RS5nS[R"XR5-nUR S5SS2S4UR S5SSS24-nSUS-US-- - nU$)a A pair wise version of the dice loss, see `dice_loss` for usage. Args: inputs (`torch.Tensor`): A tensor representing a mask labels (`torch.Tensor`): A tensor with the same shape as inputs. Stores the binary classification labels for each element in inputs (0 for the negative class and 1 for the positive class). Returns: `torch.Tensor`: The computed loss between each pairs. rr8N)sigmoidflattenr,matmulTsum)rCrD numerator denominatorrs r3pair_wise_dice_lossrN~sz^^  % %a (FELL22I**R.D)FJJrN47,CCK A +/2 2D Kr2cZURSn[R"SS9nU"U[R"U55nU"U[R "U55n[R "XB- UR5n[R "XR- SU- R5nXg-nU$)a A pair wise version of the cross entropy loss, see `sigmoid_cross_entropy_loss` for usage. Args: inputs (`torch.Tensor`): A tensor representing a mask. labels (`torch.Tensor`): A tensor with the same shape as inputs. Stores the binary classification labels for each element in inputs (0 for the negative class and 1 for the positive class). Returns: loss (`torch.Tensor`): The computed loss between each pairs. rnone reduction)shaperBCEWithLogitsLossr, ones_like zeros_likerIrJ) rCrDheight_and_width criterioncross_entropy_loss_poscross_entropy_loss_negloss_posloss_negrs r3$pair_wise_sigmoid_cross_entropy_lossr]s||A$$v6I&vuv/FG&vu/?/?/GH||2EvxxPH||2EF ~~VH  D Kr2c ^\rSrSrSrSS\S\S\S\4U4Sjjjr\R"5S\RS \RS \RS \RS \ \ \ 4 S j5r SrU=r$)EomtHungarianMatcheraaThis class computes an assignment between the labels and the predictions of the network. For efficiency reasons, the labels don't include the no_object. Because of this, in general, there are more predictions than labels. In this case, we do a 1-to-1 matching of the best predictions, while the others are un-matched (and thus treated as non-objects). cost_class cost_mask cost_dice num_pointsc>[TU]5 US:XaUS:XaUS:Xa [S5eX@lXlX lX0lg)aCreates the matcher Params: cost_class (`float`, *optional*, defaults to 1.0): Relative weight of the classification error in the matching cost. cost_mask (`float`, *optional*, defaults to 1.0): This is the relative weight of the focal loss of the binary mask in the matching cost. cost_dice (`float`, *optional*, defaults to 1.0): This is the relative weight of the dice loss of the binary mask in the matching cost. num_points (`int`, *optional*, defaults to 12544): No. of points to sample on which the mask loss will be calculated. The same set of K points are uniformly sampled for all prediction and ground truth masks to construct the cost matrix for bipartite matching. rzAll costs can't be 0N)super__init__ ValueErrorrdrarbrc)selfrarbrcrd __class__s r3rgEomtHungarianMatcher.__init__sC"  ?yA~)q.34 4$$""r2r!r mask_labels class_labelsr6c /nURSn[U5GHnX'RS5nXn USS2XG4*n X7RU 5n U SS2S4n U SS2S4n [R "SUR SU RS9n U RU RSSS5n [XSS9RS5n U RU RSSS5n[XSS9RS5n [X5n[X5nURU-URU --URU--n[R "U[R""S 55n[R$"U[R""S 55n[R&"US5n[)UR+55nUR-U5 GM UVVs/sHLunn[R."U[R0S 9[R."U[R0S 94PMN nnnU$s snnf) a Params: masks_queries_logits (`torch.Tensor`): A tensor of dim `batch_size, num_queries, num_labels` with the classification logits. class_queries_logits (`torch.Tensor`): A tensor of dim `batch_size, num_queries, height, width` with the predicted masks. class_labels (`torch.Tensor`): A tensor of dim `num_target_boxes` (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels. mask_labels (`torch.Tensor`): A tensor of dim `num_target_boxes, height, width` containing the target masks. Returns: matched_indices (`list[tuple[Tensor]]`): A list of size batch_size, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected labels (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes). rrFNrr8deviceF align_cornersg _Bg _©dtype)rSrangesoftmaxtor,randrdrprepeatrBr>r]rNrbrarcminimumtensormaximum nan_to_numrcpuappend as_tensorint64)rir!r rlrmindices batch_sizei pred_probs pred_maskra target_maskr5target_coordinatespred_coordinatesrbrc cost_matrixassigned_indicesjmatched_indicess r3forwardEomtHungarianMatcher.forwards8*,*//2 z"A-088[TU]5 [US/5 URUlX lUR Ul[R"URS-5nUR US'URSU5 URUl URUl URUl [URUR UR"URS9Ulg)a The Eomt Loss. The loss is computed very similar to DETR. The process happens in two steps: 1) we compute hungarian assignment between ground truth masks and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and mask) Args: config (`EomtConfig`): The configuration for Eomt model also containing loss calculation specific parameters. weight_dict (`dict[str, float]`): A dictionary of weights to be applied to the different losses. scipyrrF empty_weightrN)rfrgr num_labelsrno_object_weighteos_coefr,onesregister_buffertrain_num_pointsrdoversample_ratioimportance_sample_ratior_ class_weight dice_weight mask_weightmatcher)rirrrrjs r3rgEomtLoss.__init__Ms $ * ++&// zz$//A"56 == R ^\:!11 & 7 7'-'E'E$+**((((   r2sizesr6cpUSnUSSH'n[U5HupE[X$U5X$'M M) U$)Nrr) enumeratemax)rirmaxessublistindexitems r3 _max_by_axisEomtLoss._max_by_axispsCaQRyG(1 "5<6  2! r2tensorscTURUVs/sHn[UR5PM sn5n[U5/U-nUupVpxUSRn USR n [ R"XIU S9n [ R"XWU4[ RU S9n [XU 5Houp-nU SURS2SURS2SURS24RU5 SUSURS2SURS24'Mq X4$s snf)Nrrtrprr8F) rr0rSlenrtrpr,zerosrboolzipcopy_)rirr{max_size batch_shaper_heightwidthrtrppadded_tensors padding_masks padded_tensor padding_masks r3_pad_images_to_max_in_batch$EomtLoss._pad_images_to_max_in_batchxs'$$w%OwVd6<<&8w%OP7|nx/ '2$ v   ""M J#>ejjY_` 36wP]3^ /F< +FLLO+->v||A->@Q&,,q/@QQ R X XY_ `AFL*6<<?*,=fll1o,== >4_,,&PsD%r rmrc UnURupVn[R"URS9nUR U5n [ R "[X#5V VV s/sH un up{XPM sn nn 5n [ R"XV4UR[ RURS9n XU 'URSS5nU"X5nSU0nU$s sn nn f)a-Compute the losses related to the labels using cross entropy. Args: class_queries_logits (`torch.Tensor`): A tensor of shape `batch_size, num_queries, num_labels` class_labels (`list[torch.Tensor]`): List of class labels of shape `(labels)`. indices (`tuple[np.array])`: The indices computed by the Hungarian matcher. Returns: `dict[str, Tensor]`: A dict of `torch.Tensor` containing the following key: - **loss_cross_entropy** -- The loss computed using cross entropy on the predicted and ground truth labels. )weight) fill_valuertrprr8loss_cross_entropy) rSrCrossEntropyLossr$_get_predictions_permutation_indicesr,catrfullrrrp transpose)rir rmr pred_logitsr num_queriesrrXidxtargetrtarget_classes_otarget_classespred_logits_transposedloss_celossess r3 loss_labelsEomtLoss.loss_labelss"+ %0%6%6" ''t/@/@A 77@ 99-0-G H-G>66AVY-G H   %$//]h]o]o /s!,!6!6q!!<2C&0  Is C r!rlrc^TRU5nTRU5nXnTRU5upXnUSS2S4nUSS2S4n[R"5 TR UU4SjTR TRTR5n [XSS9RS5n SSS5 [UW SS9RS5n [U W U5[XU5S.n AAU $!,(df  NF=f)a$Compute the losses related to the masks using sigmoid_cross_entropy_loss and dice loss. Args: masks_queries_logits (`torch.Tensor`): A tensor of shape `(batch_size, num_queries, height, width)`. mask_labels (`torch.Tensor`): List of mask labels of shape `(labels, height, width)`. indices (`tuple[np.array])`: The indices computed by the Hungarian matcher. num_masks (`int)`: The number of masks, used for normalization. Returns: losses (`dict[str, Tensor]`): A dict of `torch.Tensor` containing two keys: - **loss_mask** -- The loss computed using sigmoid cross entropy loss on the predicted and ground truth. masks. - **loss_dice** -- The loss computed using dice loss on the predicted on the predicted and ground truth, masks. Nc&>TRU5$N)calculate_uncertainty)logitsris r3%EomtLoss.loss_masks..st99&Ar2Frqr) loss_mask loss_dice) r _get_targets_permutation_indicesrr,rsample_points_using_uncertaintyrdrrrBr>rr)rir!rlrrsrc_idxtgt_idx pred_masks target_masksrr5 point_labels point_logitsrs` r3 loss_masksEomtLoss.loss_maskss4;;GD77@)2 ::;G #,  4( #AtG, ]]_ $ D DA%%,, ! ( W\]eefghL$J0AQVW__`ab 4L,PYZ"?"JJyaPVP]P]^#FUS 2<@"#:#GH&=jj,Q1W59MSTUVWX"U\\)5::V\VcVc%dd QW~-222q9#((2,/JOOPYqrs q % "EJJyQW]WdWd$ef! ! r2Nauxiliary_predictionscURXX45nURXDSRS9n0URXXg5EUR X$U5EnUbk[ U5H\upU SnU SnUR XX45n U R5V V s0sH upU SU 3U _M n n n URU 5 M^ U$s sn n f)a This performs the loss computation. Args: masks_queries_logits (`torch.Tensor`): A tensor of shape `(batch_size, num_queries, height, width)`. class_queries_logits (`torch.Tensor`): A tensor of shape `(batch_size, num_queries, num_labels)`. mask_labels (`torch.Tensor`): List of mask labels of shape `(labels, height, width)`. class_labels (`list[torch.Tensor]`): List of class labels of shape `(labels)`. auxiliary_predictions (`dict[str, torch.Tensor]`, *optional*): if `use_auxiliary_loss` was set to `true` in [`EomtConfig`], then it contains the logits from the inner layers of the EomtMaskedAttentionDecoder. Returns: losses (`dict[str, Tensor]`): A dict of `torch.Tensor` containing three keys: - **loss_cross_entropy** -- The loss computed using cross entropy on the predicted and ground truth labels. - **loss_mask** -- The loss computed using sigmoid cross_entropy loss on the predicted and ground truth masks. - **loss_dice** -- The loss computed using dice loss on the predicted on the predicted and ground truth masks. if `use_auxiliary_loss` was set to `true` in [`EomtConfig`], the dictionary contains additional losses for each auxiliary predictions. rror!r r) r get_num_masksrprrrritemsupdate)rir!r rlrmrrrrr aux_outputs loss_dictkeyvalues r3rEomtLoss.forward=sH,,3;e&&|O)EomtLoss.get_num_masks..xsALG Lsrr)min) rKr,rrrr _shared_stater num_processesclamp)rirmrpr world_sizes r3rEomtLoss.get_num_maskstsvALAA OOIU[[P  " $ $))R/"9- )^99 KK 6A> r2)rrrrrdrrr)r'r(r)r*rdictstrrrgr0rrrr/rnparrayrr,rrrrrrrprr1rrs@r3rrLs ! z! S%Z8H! F$tCy/d3i-4<-E&RX.DY-" $* :>v, QVWYW_W_Q`  c6k  D<#ll<%,,'<rxx <  < c5<< <|2 - "ELL"U\\""5! 5! 5!  5! "' 5! 5!zAE 5#ll5$ll5%,,' 5 5<<( 5 $C$56= 5 c5<< 5n %,,  QVQ]Q]  r2rcf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)EomtPatchEmbeddingsiz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. c>[TU]5 URURp2URUR pT[ U[RR5(aUOX"4n[ U[RR5(aUOX34nUSUS-USUS--nX lX0lX@lX`l [R"XEX3S9Ul g)Nrr kernel_sizestride)rfrg image_size patch_size num_channels hidden_size isinstance collectionsabcIterable num_patchesrConv2d projection)rirr5r6r7r8r=rjs r3rgEomtPatchEmbeddings.__init__s !'!2!2F4E4EJ$*$7$79K9Kk#-j+//:R:R#S#SZZdYq #-j+//:R:R#S#SZZdYq !!} 1 5*Q-:VW=:XY $$(&))L:ir2 pixel_valuesr6cURSnX R:wa[SURSUS35eURU5R S5R SS5nU$)NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .r8)rSr7rhr?rHr)rirAr7 embeddingss r3rEomtPatchEmbeddings.forwardsx#))!, ,, ,!../yaI __\2::1=GG1M r2)r5r7r=r6r?) r'r(r)r*r+rgr,rrr1rrs@r3r0r0s. jELLU\\r2r0cr^\rSrSrSrS\SS4U4SjjrS\RS\R4Sjr S r U=r $) EomtEmbeddingsizE Construct the CLS token, mask token, position and patch embeddings. rr6Nc>[TU]5 XlURUl[R "[ R"SSUR55Ul [R "[ R"SURUR55Ul [U5UlURRn[R "UR"5UlSUR-Ul[R("X!R5UlUR-S[ R."U5R1S5SS9 g)Nr position_idsrrFF) persistent)rfrgrr6r Parameterr,randnr8 cls_tokenrnum_register_tokensregister_tokensr0patch_embeddingsr=Dropouthidden_dropout_probdropoutnum_prefix_tokens Embeddingposition_embeddingsrr expand)rirr=rjs r3rgEomtEmbeddings.__init__s   ++ekk!Q8J8J&KL!||EKK6;U;UW]WiWi,jk 3F ;++77 zz&"<"<= !"V%?%?!?#%<< =O=O#P  ^U\\+-F-M-Mg-Vchir2rAcURun nURRRRnURUR US95nUR RUSS5nURRUSS5nXPRUR5-n[R"XgU/SS9nURU5nU$)NrsrFrr)rSrQr?rrtrwrNrXrPrWrIr,rrT)rirArr target_dtyperD cls_tokensrPs r3rEomtEmbeddings.forwards*00 Aq!,,77>>DD ** ..55j"bI":":4;L;L"MM YY ZHaP \\*- r2)rNrrTrUrQr6rWrP r'r(r)r*r+rrgr,rrr1rrs@r3rGrGs>jzjdj ELL U\\  r2rGmodulequeryrrattention_maskscalingrTc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)NrF)r:rt)ptrainingrr8) r,rIrrr<rvfloat32rwrtrTrf contiguous) r_r`rrrarbrTr@ attn_weights attn_outputs r3eager_attention_forwardrks<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r2c ^\rSrSrSrU4SjrS S\RS\RS-S\\R\RS-44Sjjr S r U=r $) EomtAttentioniz=Multi-headed attention from 'Attention Is All You Need' paperc >[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gF)rfrgrr8 embed_dimnum_attention_heads num_headshead_dimrhscaleattention_dropoutrT is_causalrLineark_projv_projq_projout_projrirrjs r3rgEomtAttention.__init__s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r2Nr#rar6c URSSn/UQSPURP7nURU5RU5R SS5nUR U5RU5R SS5nUR U5RU5R SS5n[R"URR[5n U "UUUUUURURUR(dSO URS9upU R "/UQSP76R#5n UR%U 5n X4$)z#Input shape: Batch x Time x ChannelNrFrr8)rurbrT)rSrrryr rrwrxr get_interfacer_attn_implementationrkrursrfrTreshaperhrz) rir#rar@ input_shape hidden_shapequerieskeysvaluesattention_interfacerjris r3rEomtAttention.forwards6$))#2. 88b8$--8 ++m,11,?II!QO{{=)..|<FFq!L]+00>HHAN(?(M(M KK , ,.E) %8     nnJJ#}}C$,, % ! "));;;;FFH mmK0 ((r2) rrTrorrrurwrqrzryrsrxr) r'r(r)r*r+rgr,rr/rr1rrs@r3rmrms[GB./3!)||!) t+!) u||U\\D00 1 !)!)r2rmcf^\rSrSrSU4SjjrS\R S\R 4SjrSrU=r $)EomtLayerScaleir6c>[TU]5 [R"UR[ R "UR5-5Ulgr) rfrgrrLlayerscale_valuer,rr8lambda1r{s r3rgEomtLayerScale.__init__s8 ||F$;$;ejjI[I[>\$\] r2 hidden_statecXR-$rrrirs r3rEomtLayerScale.forward!sll**r2rr6N r'r(r)r*rgr,rrr1rrs@r3rrs)^+ELL+U\\++r2rinput drop_probrfcUS:XdU(dU$SU- nURS4SURS- --nU[R"X@RUR S9-nUR 5 URU5U-nU$)z[ Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). r~rr)rr)rSndimr,rxrtrpfloor_div)rrrf keep_probrS random_tensoroutputs r3 drop_pathr%s Cx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr2c^\rSrSrSrS S\S-SS4U4SjjjrS\RS\R4Sjr S\ 4S jr S r U=r $) EomtDropPathi4zXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).Nrr6c.>[TU]5 Xlgr)rfrgr)rirrjs r3rgEomtDropPath.__init__7s "r2r#cB[XRUR5$r)rrrfrir#s r3rEomtDropPath.forward;s FFr2c SUR3$)Nzp=rris r3 extra_reprEomtDropPath.extra_repr>sDNN#$$r2rr)r'r(r)r*r+rrgr,rrr,rr1rrs@r3rr4sQb#%$,#$##GU\\GellG%C%%r2rcf^\rSrSrSU4SjjrS\R S\R 4SjrSrU=r $)EomtMLPiBr6cz>[TU]5 UR=p#[URUR-5n[ R "X$SS9Ul[UR[5(a[URUl OURUl [ R "XCSS9Ul g)NTbias)rfrgr8r mlp_ratiorrvfc1r9 hidden_actr,r activationfc2rir in_features out_featureshidden_featuresrjs r3rgEomtMLP.__init__Cs %+%7%77 f0063C3CCD99[E f'' - -$V%6%67DO$//DO99_Fr2rclURU5nURU5nURU5nU$r)rrrrs r3rEomtMLP.forwardNs2xx - |4 xx - r2)rrrrrrs@r3rrBs) GELLU\\r2rcf^\rSrSrSU4SjjrS\R S\R 4SjrSrU=r $) EomtSwiGLUFFNiUr6c$>[TU]5 UR=p#[URUR-5n[US-S- 5S-S-S-n[ R "USU-SS9Ul[ R "XCSS9Ulg)Nr8rTr) rfrgr8rrrrv weights_in weights_outrs r3rgEomtSwiGLUFFN.__init__Vs %+%7%77 f0063C3CCD2Q67!;AAE))K_1D4P99_Nr2rcURU5nURSSS9up#[RR U5U-nUR U5$)Nr8rFr)rchunkrr<silur)rirx1x2hiddens r3rEomtSwiGLUFFN.forward_sQ|4 ##A2#.##B'",''r2)rrrrrs@r3rrUs)O(ELL(U\\((r2rc^\rSrSrSrS\SS4U4SjjrS S\RS\RS-S\R4S jjr S r U=r $) EomtLayerifzCThis corresponds to the Block class in the original implementation.rr6Nc>[TU]5 [R"URUR S9Ul[U5Ul[U5Ul URS:a[UR5O[R"5Ul[R"URUR S9UlUR (a[#U5UlO['U5Ul[U5Ulg)Nepsr~)rfrgr LayerNormr8layer_norm_epsnorm1rm attentionr layer_scale1drop_path_raterIdentityrnorm2use_swiglu_ffnrmlpr layer_scale2r{s r3rgEomtLayer.__init__is \\&"4"4&:O:OP &v.*62@F@U@UX[@[f&;&;>2D#U  $ 1 12G H'<= M zz-0 xx - ((6 ~~l3mC r2)rrrrrrrrr^rs@r3rrfsUM3z3d3&/3|| t+  r2rcf^\rSrSrSU4SjjrS\R S\R 4SjrSrU=r $)EomtLayerNorm2dic">[TU]XUS9 g)N)relementwise_affine)rfrg)rir7raffinerjs r3rgEomtLayerNorm2d.__init__s 6Jr2rr6cURSSSS5n[R"XRURUR UR 5nURSSSS5nU$)Nrr8rr)permuteF layer_normnormalized_shaperrrrs r3rEomtLayerNorm2d.forwards`#++Aq!Q7 ||L2G2GVZV_V_aeaiaij #++Aq!Q7 r2r&)gư>Trrs@r3rrs)KELLU\\r2rcj^\rSrSrS\4U4SjjrS\RS\R4SjrSr U=r $)EomtScaleLayerirc >[TU]5 URn[R"X"SSS9Ul[ URUl[R"UUSSUSS9Ul [U5Ul g)Nr8r2rrF)r3paddinggroupsr) rfrgr8rConvTranspose2dconv1r rrr>conv2r layernorm2drirr8rjs r3rgEomtScaleLayer.__init__ss (( '' aXYZ  !2!23YY     +;7r2r#r6cURU5nURU5nURU5nURU5nU$r)rrrrrs r3rEomtScaleLayer.forwardsB =1  6  =1 ((7 r2)rrrr r'r(r)r*rrgr,rrr1rrs@r3rrs/8z8 U\\ellr2rcj^\rSrSrS\4U4SjjrS\RS\R4SjrSr U=r $)EomtScaleBlockirc>[TU]5 URUl[R "[ UR5Vs/sHn[U5PM sn5Ulgs snfr) rfrgnum_upscale_blocks num_blocksr ModuleListrurblockrirrrjs r3rgEomtScaleBlock.__init__sM  33]]E$//DZ#[DZqN6$:DZ#[\ #[sA*r#r6c<URH nU"U5nM U$r)r)rir#rs r3rEomtScaleBlock.forwards ZZE!-0M r2)rrrrs@r3rrs1]z] U\\ellr2rcj^\rSrSrS\4U4SjjrS\RS\R4SjrSr U=r $) EomtMaskHeadirc >[TU]5 URn[R"X"5Ul[R"X"5Ul[R"X"5Ul[URUl gr) rfrgr8rrvrrfc3r rrrs r3rgEomtMaskHead.__init__s[ (( 99[699[699[6 !2!23r2r#r6cURURU55nURURU55nURU5nU$rrrrr rs r3rEomtMaskHead.forwardsD(?@ (?@ / r2r rrs@r3rrs/4z4U\\ellr2rc\rSrSr%Sr\\S'SrSrSr Sr S/r S r \ \S .r\R""5S \R&S S 4Sj5rSrg )EomtPreTrainedModelizz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. reomtrA)imageFrT)r#r$r_r6NclURRn[U[R[R [R 45(a[R"UR[R"S5S9 URbz[RRRUR5up4US:aS[R"U5- OSn[R"URU*U5 gg[U[R 5(aA[R""UR5 [R$"UR5 g[U[R&5(ay[R("URSSS9 UR*bK[-URSS5(d.[R$"URUR*5 ggg[U[.5(aH[1US 5(a6[R2"UR4URR65 gg[U[85(a[R:"UR<SUS9 [R$"UR>5 [R@"URB[RD"URBRFS 5RIS 55 g[U[J5(aT[RL"URNS-5nURPUS '[R@"URRU5 g[U[T5(a![R""URV5 gg) N)arrr~)rstd_is_hf_initializedFrrFrJ),rinitializer_ranger9rrvr>rinitkaiming_uniform_rmathsqrtrr,_calculate_fan_in_and_fan_outuniform_rones_zeros_rVnormal_ padding_idxgetattrrhasattr constant_rrrG trunc_normal_rNrPrrIr rSrXrrrrrEomtForUniversalSegmentationattn_mask_probs)rir_rfan_inrboundrs r3 _init_weights!EomtPreTrainedModel._init_weightssckk++ fryy"))R5G5GH I I  ! !&--499Q< @{{&!HHMMGG V 17!DIIf-- fkkE659' - - JJv}} % KK $  - - LLSa 8!!-gfmmMach6i6i FMM&*<*<=>7j-  / /vy))v~~t{{/K/KL*  / /   v//cs C KK.. / JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h  ) ) ::f&7&7!&;r2r&)r'r(r)r*r+rr.base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_no_split_modules_supports_sdparrm_can_record_outputsr,rrModuler+r1r&r2r3rrsm $O!&+#$ N"#  ]]_/BII/$//r2rzV The EoMT Model with head on top for instance/semantic/panoptic segmentation. c2^\rSrSrSrS\4U4SjjrS\S\S\S\S \\ \4S \\ \44 S jr S \\ \4S \4S jr \ \ \SS\S\\S-S\\S-S\\S-S\\S \4 Sjj555rSrS\R4Sjr\S5rSrU=r$)r'irArc>[TU]U5 XlURUl[ U5Ul[ R"URURS9Ul [ R"URUR5Ul [ R"[UR5Vs/sHn[!U5PM sn5Ul[%U5Ul[)U5Ul[ R,"URUR.S-5UlUR2UR4-UR2UR4-4UlUR8UR:UR<S.Ul[AXR>S9Ul!URES[FRH"URJ55 URM5 gs snf)Nrr)rrr)rrr()'rfrgrnum_hidden_layersrGrDrrr8r layernormrVrr`rrurlayersr upscale_blockr mask_headrvrclass_predictorr5r6 grid_sizerrrrrrXrr,rr post_initrs r3rg%EomtForUniversalSegmentation.__init__ sr   !'!9!9(0f&8&8f>S>ST\\&"4"4f6H6HI mmfF^F^@_$`@_1Yv%6@_$`a +F3%f-!yy););V=N=NQR=RS ++v/@/@@&BSBSW]WhWhBhi"("5"5++++.  "=M=MN . 6;L;L0MN %%as=G*r!r rlrmrr6cURUUUUUS9nURR5H)upxUR5HupXy;dM X-n M M+ U$)Nr!r rlrmr)rXrr) rir!r rlrmrrrrloss_keyrs r3 get_loss_dict*EomtForUniversalSegmentation.get_loss_dict(sj(,~~!5!5#%"7 (6(  ++113KC"+//"3?ND#44 r2rc4[UR55$r)rKr)rirs r3get_loss%EomtForUniversalSegmentation.get_loss@s9##%&&r2Nr%r@c SupgSnUc [S5eURU5n [UR5GHupXRUR R - :XamURRSSS2SS24RU RSSS5RU R5n [R"X4SS9n XRUR R - :GaUR(d7UR XR- UR R -S:GaUR#U 5n UR%U 5upXn4- nX4- n[R&"U RSU RSU RSU R[R(S9n[*R,"XR.S S 9nUR1UR3S5UR3S5S5nUR R4nUURR6-nUS:USS2SU2US24'UR9UUR XR- UR R -UUURS 9nUSS2SS 4RSUR R:SS5nUR=5R?U)S 5nU "X5n GM UR#U 5nUR%U5upXn4- nX4- nSnUb@Ub=Sn[AXg5H,upURCUUUUSS9nUUREU5- nM. [GUUUUUS9$)a' mask_labels (`list[torch.Tensor]`, *optional*): list of mask labels of shape `(num_labels, height, width)` to be fed to a model class_labels (`list[torch.LongTensor]`, *optional*): list of target class labels of shape `(num_labels, height, width)` to be fed to a model. They identify the labels of `mask_labels`, e.g. the label of `mask_labels[i][j]` if `class_labels[i][j]`. patch_offsets (`list[torch.Tensor]`, *optional*): list of tuples indicating the image index and start and end positions of patches for semantic segmentation. )r&r&Nz You have to specify pixel_valuesrrFrr)rprtbilinear)sizemode)probnum_query_tokensencoder_start_tokensrp.ger~rA)rr!r r"r%)$rhrDrr9r7rrr`rrXrSrwrpr,rrfr(r8predictrrr interpolater=r rJrrU_disable_attention_maskrpr masked_fillrrCrFr)rirArlrmr%r@masks_queries_logits_per_layerclass_queries_logits_per_layerrar#r layer_moduler`norm_hidden_statesr!r interpolated_logitsrMrNsequence_outputrrs r3r$EomtForUniversalSegmentation.forwardCs*JPF&  ?@ @ 5 !*4;;!7 C,,t{{/E/EEE ))$1*5<<]=P=PQR=SUWY[\__`m`t`tu % 5*@a H ,,t{{/E/EEE !5!5c&>',,Q/1D1I1I!1Lb'#$(;;#:#: '7$//:[:['[$ObdeNeq"3#3"35I5JJK"&!=!="--c4J4J.JT[[McMc.cd%5)=)00 ">""04 !=!D!DRIhIhjlnp!q!/!5!5!7!C!C^OUY!Z(GM]"8`..759\\/5R2&*AA&&*AA&  "|'?D>A.?:$!..)=)= +!-*. /  i00?2!5!5-'   r2c.URR$r)rDrQrs r3get_input_embeddings1EomtForUniversalSegmentation.get_input_embeddingss///r2rcUSS2SURR2SS24nURU5nUSS2URRURR-S2SS24nUR SS5nUR "URSS/URQ76nURU5nURU5n[R"SX$5nXS4$)Nrr8rrFzbqc, bchw -> bqhw) rrr<rDrUrrrSr=r;r:r,einsum)rir query_tokens class_logits prefix_tokens mask_logitss r3rO$EomtForUniversalSegmentation.predictsa!:4;;#:#:!:A=> ++L9 q$++"9"9DOO<]<]"]"_abbc %//15 %--m.A.A!.DbZ4>>Z ~~l3 **=9 ll#6 T ((r2c~US:a6[R"URSX$S9U:nSUSS2SU2US24U'U$)Nrrro)r,rxrS) attn_maskrLrMrNrprandom_queriess r3rQ4EomtForUniversalSegmentation._disable_attention_masksT !8"ZZ (: Rr2) r<rrXrDr=r8r9r;r7r`r:r)NNN)r'r(r)r*r.rrgrr+r,rCrFrrrr0rrrrr[r,rO staticmethodrQr1rrs@r3r'r's? %Oz8$%    $CK0  c6k 0'$sF{"3'' ,0,0-1 e e &\D(e 6lT) e F|d* e +, e  ,e  e N0)ell) r2r')F)r~)r~F)Lcollections.abcr:rr dataclassesrnumpyr-r,torch.nn.functionalrr<rrrr activationsr file_utilsr r r modeling_layersr modeling_utilsrrprocessing_utilsrutilsrrr utils.genericrutils.output_capturingrconfiguration_eomtrscipy.optimizer accelerateraccelerate.utilsrrrBrNr]r4r_rrrrr0rGrrkrmrrrrrrrrrrrrrr'__all__r&r2r3r{s"* $! &!LL9F&PP75*4''  44  4BLQLL5:\\ \\@6,u||X]XdXd8g299gTff<u||U\\VY^c^j^j(uryyup "))B"RYY"X% II% <<% % << % LL4' %  %%.8)BII8)v+RYY+ U\\ e T V[VbVb  %299 %bii&(BII("'*'TbllRYY2 RYY 299"///////d @#6@  @F !"@ Ar2