EADST

Quick Review: Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

作者：XD / 发表： 2023年12月6日 23:44 / 更新： 2023年12月6日 23:52 / 科研学习 / 阅读量：165

Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

Steps for Implementation:

  1. Generate Data: Prepare and preprocess the dataset suitable for training the model.
  2. GPTQ: Apply GPTQ method for optimizing the quantization precision of model parameters.
  3. Train LayerNorm Only: Focus on training the Layer Normalization component of the model for fine-tuning and optimization.

本文作者：XD 转载请标明出处：http://www.eadst.com/blog/223

本站采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

上一篇
Review: H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
下一篇
Quick Review: Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
相关标签
LLM Quantization
About Me
XD
Goals determine what you are going to be.
Category
标签云
GIT Streamlit FlashAttention LeetCode Website Ptyhon SQL diffusers Interview Windows Diagram uwsgi Bert Review Tracking Cloudreve JSON Domain BeautifulSoup Plate Paper Github ONNX VPN tar DeepStream Statistics TensorRT Baidu Translation HuggingFace Shortcut Proxy WebCrawler LoRA CV TensorFlow Gemma Jetson Numpy YOLO Zip Food Tensor 搞笑 报税 Quantize Password Linux TSV Pickle transformers UNIX ChatGPT Template Ubuntu Qwen Permission Pillow NLP 域名 logger Paddle 阿里云 OpenCV XGBoost Bin QWEN Image2Text Dataset 腾讯云 NameSilo Distillation Attention Knowledge Quantization scipy Bipartite 关于博主 Nginx uWSGI Bitcoin VGG-16 GoogLeNet Web Datetime C++ AI PIP Markdown Anaconda Michelin Pytorch Git CEIR Augmentation ResNet-50 Python Use v2ray GPT4 CSV Video Heatmap Crawler LLAMA ModelScope SVR XML Google CTC LaTeX NLTK SPIE Random Land HaggingFace InvalidArgumentError RGB Data Excel Conda Clash 签证 SQLite COCO Disk Input OpenAI GPTQ PyCharm CUDA GGML Safetensors Tiktoken Color Sklearn Django OCR Docker torchinfo CAM BTC Pandas Hungarian PDF Plotly Algorithm EXCEL LLM Base64 llama.cpp
站点统计

本站现有博文242篇,共被浏览292078次

本站已经建立1782天!

热门文章
文章归档
回到顶部