ZjSSKJr SSKJr SSKJrJr SSKJr SSK J r J r \R"\ 5r\"5(a SSKrSSKJr S SSjjr"S S \5r"S S \5rg)) annotations)Any)is_torch_availablelogging) ConversionOps)get_module_from_nameshould_convert_moduleNc SSKJn Uc/n[UR55Hupx[ U[ R 5(dM&[Xq5(dM8URS5upn U (aURU 5OUn U"U(d UROSU(d UROSU(dURSLOSUUUSS9n [XU 5 M U$)a Replace nn.Linear modules with empty SINQLinear modules. Args: model: The model to modify modules_to_not_convert: List of module names to skip quant_config: SINQ quantization config dict (None for pre-quantized models) compute_dtype: Computation dtype for the quantized layers device: Device string for the quantized layers pre_quantized: Whether loading a pre-quantized checkpoint Returns: The modified model with SINQLinear modules r) SINQLinearN.FT) in_features out_featuresbias quant_config compute_dtypedeviceuse_unpack_kernel)sinq.sinqlinear_hfr list named_modules isinstancennLinearr rpartition get_submodulerrrsetattr)modelmodules_to_not_convertrrr pre_quantizedr full_namemodule parent_path_ child_nameparent sinq_layers o/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/integrations/sinq.pyreplace_with_sinq_linearr) s,.%!#!%"5"5"78 &")),, $YGG %.%9%9#%>" 5@$$[1e2?**T4A,,t2?&++T)U%'"  J/'9* Lc@\rSrSrSrSrSSSjjrSrg) SinqQuantizeSa Param-level ConversionOp for SINQ (from FP weights). At load time, for each `Linear.weight` that should be quantized: - The SINQLinear module already exists (created in _process_model_before_weight_loading) - We just call quantize() on it with the loaded weight tensor cXlgN hf_quantizerselfr1s r(__init__SinqQuantize.__init__\(r*Nc [[UR555upg[U[5(aUSOUn[ X#5upU R U5 UbURU5 SU l0$)NrT) nextiteritemsrrr quantizediscard_is_hf_initialized) r3 input_dictrfull_layer_name missing_keyskwargsr$values weight_tensorr" tensor_names r(convertSinqQuantize.convert_soj..012 %/%=%=q 6 25J &  #   1$(! r*r0)NNNr>zdict[str, Any]rztorch.nn.Module | Noner?z str | Nonereturnzdict[str, torch.Tensor]__name__ __module__ __qualname____firstlineno____doc__r4rE__static_attributes__r*r(r,r,SsG) )-&* "&$  !r*r,c>\rSrSrSrSrSSSjjrSrg) SinqDeserializeva ConversionOp for loading *pre-quantized* SINQ checkpoints. Checkpoint layout (what `SINQLinear.state_dict` produces) is, per module: .W_q .bias .meta WeightConverter in the quantizer is configured so that: - we group ".W_q", ".meta", ".bias" as input_dict - conceptually treat them as belonging to ".weight" - and call this SinqDeserialize.convert to load the state into the existing SINQLinear. The returned dict is {} because we load directly into the module. cXlgr/r0r2s r(r4SinqDeserialize.__init__r6r*Nc [UR55H#upV[U[5(dMUSX'M% URS5nURS5nURS5n UbUc?[ [ UR 555n[U[5(aUSnX60$[X#5upUUS.n U bXS'U RU 5 SU l 0$)Nrz.W_qz.metaz.bias)W_qmetarT) rr:rgetr8r9rBr load_state_dictr=) r3r>rr?rAkvrWrXrr"r$states r(rESinqDeserialize.converts))+,DA!T"" !! -nnV$~~g&~~g& ;$,T*++-./A!T""aD#' '(@     &Mu%$(! r*r0)NNrGrIrPr*r(rRrRvsD ) )-&* #"#&#$ # ! ##r*rR)NNNzcuda:0F)rtorch.nn.Modulerzlist[str] | Nonerz dict | Nonerz torch.dtyperstrr boolrHr_) __future__rtypingrtransformers.utilsrrcore_model_loadingrquantizers.quantizers_utilsr r get_loggerrJloggertorchtorch.nnrr)r,rRrPr*r(rks#:.U   H % 04 $!% 0 0,00 0  0  00f = F7m7r*