EADST

Quick Review: Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

作者：XD / 发表： 2023年12月6日 23:44 / 更新： 2023年12月6日 23:52 / 科研学习 / 阅读量：740

Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

Steps for Implementation:

  1. Generate Data: Prepare and preprocess the dataset suitable for training the model.
  2. GPTQ: Apply GPTQ method for optimizing the quantization precision of model parameters.
  3. Train LayerNorm Only: Focus on training the Layer Normalization component of the model for fine-tuning and optimization.

本文作者：XD 转载请标明出处：http://www.eadst.com/blog/223

本站采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

上一篇
Review: H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
下一篇
Quick Review: Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
相关标签
LLM Quantization
About Me
XD
Goals determine what you are going to be.
Category
标签云
Docker Hungarian WebCrawler YOLO Ptyhon Attention Sklearn Plate Pandas 算法题 VGG-16 FP32 HuggingFace Land 腾讯云 BeautifulSoup GPT4 TSV SVR PyCharm Streamlit MD5 ModelScope Markdown uWSGI PDF Translation Baidu Bert Data Pillow Git Image2Text hf Vmess uwsgi Clash Excel llama.cpp CV Bitcoin Magnet Pytorch Paper VSCode scipy RGB Bipartite Nginx Template Food Augmentation CSV Python tar C++ Zip NLP Base64 Domain Random Conda GoogLeNet GIT Qwen Google FastAPI 阿里云 FP64 EXCEL Password Datetime BF16 Tiktoken LeetCode Transformers Algorithm Jetson ResNet-50 Statistics Card FlashAttention Heatmap PIP AI Bin Video XML Github COCO Quantization CUDA Shortcut transformers LaTeX logger Paddle Input v2ray DeepStream CTC Hotel Website Permission diffusers Use Windows Plotly XGBoost SQLite Quantize Safetensors TensorRT CEIR OpenCV Color Tracking Review UNIX SQL Gemma Crawler Interview Math Michelin Diagram Disk torchinfo OCR VPN TensorFlow LLM 报税 git-lfs Django GGML Ubuntu Dataset Proxy Qwen2.5 Linux Hilton Web InvalidArgumentError 签证 域名 LoRA git BTC Numpy ONNX Knowledge Distillation OpenAI Cloudreve Tensor GPTQ Pickle FP16 JSON 关于博主 QWEN NLTK Anaconda LLAMA ChatGPT HaggingFace NameSilo Vim SPIE CAM 搞笑
站点统计

本站现有博文266篇,共被浏览440727次

本站已经建立2019天!

热门文章
文章归档
回到顶部