Carl Franzen / VentureBeat:

Huawei's Zurich Lab unveils SINQ, an open-source quantization method that it claims can reduce LLM memory use by 60-70% without significant quality loss  —  - Dual-Axis Scaling: Instead of using a single scale factor for quantizing a matrix, SINQ uses separate scaling vectors for rows and columns.

Carl Franzen / VentureBeat:

Huawei's Zurich Lab unveils SINQ, an open-source quantization method that it claims can reduce LLM memory use by 60-70% without significant quality loss  —  - Dual-Axis Scaling: Instead of using a single scale factor for quantizing a matrix, SINQ uses separate scaling vectors for rows and columns.

Source: TechMeme
Source Link: http://www.techmeme.com/251004/p18#a251004p18