EADST

Quick Review: Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

作者：XD / 发表： 2023年12月6日 23:44 / 更新： 2023年12月6日 23:52 / 科研学习 / 阅读量：1922

Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

Steps for Implementation:

  1. Generate Data: Prepare and preprocess the dataset suitable for training the model.
  2. GPTQ: Apply GPTQ method for optimizing the quantization precision of model parameters.
  3. Train LayerNorm Only: Focus on training the Layer Normalization component of the model for fine-tuning and optimization.

本文作者：XD 转载请标明出处：http://www.eadst.com/blog/223

本站采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

上一篇
Review: H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
下一篇
Quick Review: Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
相关标签
LLM Quantization
About Me
XD
Goals determine what you are going to be.
Category
标签云
强化学习 VPN Markdown Linux Pytorch Mixtral diffusers TSV Proxy FP32 Rebuttal 腾讯云 公式 NameSilo HuggingFace Ubuntu OCR Sklearn TensorFlow Color VGG-16 llama.cpp mmap 净利润 证件照 ONNX 继承 Cloudreve Gemma 域名 Input RGB Quantize LoRA Jetson Dataset 阿里云 Bin VSCode Bitcoin hf 关于博主 Numpy TTS FP8 FlashAttention Safetensors InvalidArgumentError Windows Conda Bipartite Michelin tqdm 多线程 transformers Qwen2.5 Vim Translation Quantization Random uWSGI GPT4 Nginx Ptyhon API 版权 CTC Datetime 顶会 Hungarian Augmentation Hotel 财报 Baidu XML Card Jupyter XGBoost 图形思考法 PDF SQLite 报税 SVR Docker Paper LeetCode 云服务器 Web Tiktoken SAM UNIX Llama Search COCO SPIE CUDA Food Password Animate Transformers Permission Template Image2Text NLTK Crawler Base64 AI Attention Shortcut Zip Bert Plotly Plate PyTorch Heatmap Algorithm JSON 搞笑 BF16 Pillow BeautifulSoup Qwen QWEN OpenAI Breakpoint printf ResNet-50 Video DeepStream CV IndexTTS2 Excel CAM scipy CSV Knowledge Anaconda RAR Website Use SQL Github Data Diagram Review Google LaTeX Land Distillation Disk Streamlit YOLO Python LLM GPTQ GoogLeNet icon CEIR Paddle FP64 Logo v2ray torchinfo Tracking OpenCV Qwen2 FP16 git GGML Firewall News GIT Vmess WebCrawler Math EXCEL Agent uwsgi CLAP Domain HaggingFace Interview 算法题 Git 递归学习法 Freesound Statistics tar Pickle PyCharm 多进程 WAN git-lfs ChatGPT 签证 Clash UI NLP logger 第一性原理 DeepSeek Pandas Magnet Tensor TensorRT BTC v0.dev PDB Miniforge 飞书 LLAMA MD5 CC Hilton 图标 ModelScope FastAPI PIP Django 音频 C++ Claude
站点统计

本站现有博文324篇,共被浏览807414次

本站已经建立2508天!

热门文章
文章归档
回到顶部