QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models
Paper: https://arxiv.org/abs/2310.09259
Code: https://github.com/IST-DASLab/QUIK
Organization: ETH Zurich
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
Paper: https://arxiv.org/abs/2306.03078
Code: https://github.com/Vahe1994/SpQR
Organization: University of Washington
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of Large Language Models
Paper: https://arxiv.org/abs/2309.05516
Code: https://github.com/intel/neural-compressor
Organization: Intel
Norm Tweaking: High-performance Low-bit Quantization of Large Language Models
Paper: https://arxiv.org/abs/2309.02784
Code: None
Organization: Meituan
Review: H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Level: Average, Not Recommend
Check All Values from One Tensor Equal to One Value
Pytorch GPTQ Dequantizing Function
Pytorch Q4_1 Quantize and Dequantize aligning with llama.cpp
Pytorch Q4_0 Quantize and Dequantize aligning with llama.cpp
QWEN7B to LLAMA GPTQ model structure
QWEN7B to LLAMA7B Model Structure
GGML Q4_0 Quantize Analysis in llama.cpp
Check the KB Size of the File or Folder in the Linux
Qwen-7B-Chat模型结构注释
Update Code in Django+Nginx+uwsgi Environment
from diffusers.utils import randn_tensor ImportError: cannot import name 'randn_tensor' from 'diffusers.utils'
本站现有博文322篇,共被浏览792639次
本站已经建立2489天!