Quantization¶
4-bit (Int4) group quantization: quantized weight container, dequantization,
and the quantized linear/matmul path. This is what lets the supported models
run as -4bit MLX checkpoints.
4-bit (Int4) group quantization: quantized weight container, dequantization,
and the quantized linear/matmul path. This is what lets the supported models
run as -4bit MLX checkpoints.
anllm.core.quantize.QuantizedWeights
¶anllm.core.quantize.quantized_linear(x, w, bias=None)
¶anllm.core.quantize.dequantize_linear(mx_layer)
¶