Zj7SSKJr SSKJr SSKJr SSKJrJr SSK J r J r J r J r Jr \ "5(a SSKrSSKJr \ "5(aSS KJr \ "5r\ "5(a \(dSSKr\R,"\5r"S S \5r"S S \R R45r"SS\R85r\"SS9S5rSS\\ S-4Sjjr!g)) lru_cache)ACT2FN) ConversionOps)get_module_from_nameshould_convert_module)is_accelerate_availableis_fbgemm_gpu_availableis_torch_availableis_torch_xpu_availableloggingN)nn)init_empty_weightsc \rSrSrSrS S\\\R\ \R-4S\RRS-S\\\R44Sjjr Sr g) FbgemmFp8Quantize,cXlgN hf_quantizer)selfrs u/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/integrations/fbgemm_fp8.py__init__FbgemmFp8Quantize.__init__-s(N input_dictmodelreturnc z[UR55SupEUSnSSKJn [ X$5upx[ Xv5(GaUS:Xa}UR SS5n U Rn U RSU S5n [U 5upU RU 5nUR SS5nU RU SSU S5nOUS:Xa|UR SS5n U Rn U RSU S5n [U 5upU RU 5nUR SS5nU RU SU SS5nOI[U5up[RRURURSS55nU[RRW5US3W0$) Nrr)FbgemmFp8Llama4TextExperts gate_up_proj down_proj_scale)tupleitems integrationsr r isinstance transposeshapereshapequantize_fp8_per_rowtorchr Parameterview)rrrkwargs target_keyvaluer module tensor_nametransposed_paramoriginal_shapeflattened_paramnew_value_flatweight_scale_flat new_value weight_scales rconvertFbgemmFp8Quantize.convert0s "*"2"2"45a8 a=25E f 9 9n,$)??1a#8 "2!7!7"2":":2~b?Q"R5I4Y1+22>B %//15 0889JA~^_O`a  +$)??1a#8 "2!7!7"2":":2~b?Q"R5I4Y1+22>B %//15 0889JN[\L]_`a &:5&A #I 88--l.?.? @R@RST@UWX.YZLEHH..y9j\;PR^__rrr)__name__ __module__ __qualname____firstlineno__rdictstrr.TensorlistrModuler=__static_attributes__rrrr,sn) )-2`ellT%,,-???@2`xx%2` c5<< 2`2`rrcH^\rSrSr\R 4U4SjjrSrSrU=r $)FbgemmFp8Linearec^>[TU]XU5 XlX l[R R [R"X!4US95Ul[R R [R"US4[RS95Ul URS[R"S/[RS9SS9 U(aP[R R [R"UR[RS95Ul gSUl g)Ndtyper"input_scale_ubF persistent)superr in_features out_featuresr.rr/zerosweightfloat32r<register_bufferfloatbias)rrTrUr[rO __class__s rrFbgemmFp8Linear.__init__fs D9&(hh((l5PX])^_ !HH..u{{L!;LTYTaTa/bc -u{{A3ekk/R_de **5;;8I8IRWR_R_+`aDIDIrc /URSSQSP7n[URSURS5R5URS9up4UR R [R5n[(ac[R"UURR5URS5UR5URURS9nOP[R R"R%X0RXESS9nURbX`R-OUnUR UR&5nUR)U5nAAU$)Nr#)scale_ub)scale_ascale_b out_dtyper[Tuse_fast_accum)r+r-r0 contiguousrPr<tor.rX_is_torch_xpu_available _scaled_mmrWt unsqueezerOr[opsfbgemmf8f8bf16_rowwisedevicer,)rx output_shape x_quantizedx_scaleweight_scale_float32outputs rforwardFbgemmFp8Linear.forwardts**"*r*  4AFF2qwwr{4K4V4V4Xcgcvcvw $0033EMMB " "%% ))"-,..0''YY FYY%%66[['X\7F,099+@Vii'fF188$ -  r)r[rTrUrWr<) r?r@rArBr. float8_e4m3fnrrurH __classcell__r\s@rrKrKes>C>Q>Q rrKcH^\rSrSr\R 4U4SjjrSrSrU=r $)r cT>[TU]5 URUlURUlUR UlURUl[URUl [RR[R"URUR SUR -4[RS95Ul[RR[R"URSUR S-4[R S95Ul[RR[R"URUR UR 4[RS95Ul[RR[R"URUR S4[R S95UlUR)S[R"S/[R*S9SS9 g)NrrNr"rPFrQ)rSrnum_local_experts num_expertsintermediate_size hidden_size expert_dimr hidden_actact_fnr.rr/rVrwr!rXgate_up_proj_scaler$down_proj_scalerYrZ)rconfigrOr\s rr#FbgemmFp8Llama4TextExperts.__init__s !33!'!9!9!--00V../ !HH.. KK))4+;+;Q=PQY^YlYl m #((("4"4 KK))1doo.AB%-- X# ++ KK))4??DPQVVWY[\]hhjlln+11 %%$%%a(221a89K9LMXXZ\\^(222669!<=O=PQVVWY[\]hhjlln+11 yy''88$%%a(221a89L:LMXXZ .1!45H6HINNrSTU``b#' 9YY%%66$%%a(221a89K9LMXXZ .1!45G5HINNrSTU``b#' 7[[..I3G _c_r_r3s 0 &*&:&:&=&=emm&L #&& % 0 0'NN1%//15@@B+55b93A6;;BBMMOQQS+11 ! !& 0 0 A A'NN1%//15@@B#+A.33B:EEG#' !B! +KNA)B"nn]%9%9: D$4$455r) rr$rrr!rrrr~) r?r@rArBr.rXrrurHrxrys@rr r s%*]]f0P6P6rr r")maxsizec[(aSSKJn U"S5R$[R R R$)Nr" get_kernelzkernels-community/fp8-fbgemm)rg hub_kernelsrr-r.rkrlrs rget_quantize_fp8_per_rowrs3+89NNN 99   0 00rmodules_to_not_convertc[5qSnU(a0OSS0nUR5Hupx[Xq5(dMSn [ SS9 UR R S:Xa@[URSUR5n [U =(d UR5n O^[U[R5(a?[URURUR SL40UD6n U R#S5 SSS5 U cMUR%Xy5 SnM U(d[&R)S5 U$!,(df  NH=f) a A helper function to replace all `torch.nn.Linear` modules by `FbgemmFp8Linear` modules. This will enable running your models using high performance fp8 kernel from FBGEMM library. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. modules_to_not_convert (`list[`str`]`, *optional*, defaults to `None`): Names of the modules to not convert. In practice we keep the `lm_head` in full precision for numerical stability reasons. quantization_config (`FbgemmFp8Config`): The quantization config object that contains the quantization parameters. pre_quantized (`book`, defaults to `False`): Whether the model is pre-quantized or not FrONT)include_buffersLlama4TextExperts text_configzYou are loading your model using FP8 quantization but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.)rr- named_modulesrrr\r?getattrrr r)rLinearrKrTrUr[requires_grad_ set_submoduleloggerwarning) rrquantization_config pre_quantizedtp_planhas_been_replaced module_kwargs module_namer4 new_modulers rreplace_with_fbgemm_fp8_linearr s.$45'Bgt_M$224 $[II   5((,?? &ellM5<<P 7 8Su||T FBII..,&&''KKt+$  ))%06"    K4 5 58   L=6 5s B9E E )NNFN)" functoolsr activationsrcore_model_loadingrquantizers.quantizers_utilsrrutilsr r r r r r.r acceleraterrgfbgemm_gpu.experimental.gen_ai fbgemm_gpu get_loggerr?rrrrKrGr rrFrDrrIrrrs .U-02%<)   H %6` 6`r,ehhoo,^i6i6X 111tx:#'9t#3:r