EADST

Quick Review: QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models

作者：XD / 发表： 2023年12月7日 00:06 / 更新： 2023年12月7日 00:06 / 科研学习 / 阅读量：2292

QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models

Key Features:

  • Int4 Calculation: Implements 4-bit integer (Int4) calculations to significantly enhance inference speed.
  • Reduced KV Cache Memory: Utilizes this technique mayb decrease Key-Value (KV) cache memory requirements, enabling more efficient processing of large language models.

本文作者：XD 转载请标明出处：http://www.eadst.com/blog/226

本站采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

上一篇
Quick Review: SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
下一篇
Quick Review: ZeroQuant-FP
相关标签
LLM Quantization
About Me
XD
Goals determine what you are going to be.
Category
标签云
TTS UI EXCEL LLAMA Password Bitcoin Base64 printf BF16 ms-swift Vim Domain Transformers Pillow Python API icon 音频 Data Augmentation transformers XML HaggingFace Qwen2 Django 公式 IndexTTS2 Card CV Clash JSON logger Random 递归学习法 Website Template 论文 Baidu Bipartite MD5 BeautifulSoup DeepStream 域名 RGB GIT Mixtral Tracking Numpy FP32 证件照 Hungarian ChatGPT BTC uwsgi torchinfo FP64 WAN Quantization RL 多进程 PyTorch OpenAI Logo GoogLeNet Windows 图标 SVR TSV Paddle Permission 报税 ResNet-50 GPT4 Plate tqdm Claude NLP ModelScope Diagram 第一性原理 Heatmap Image2Text FP16 Docker Jupyter Algorithm CAM diffusers Proxy 算法题 v0.dev Zip CEIR Qwen2.5 git-lfs FastAPI 图形思考法 GPTQ 飞书 Anaconda RAR FlashAttention scipy Llama Jetson OCR Land Markdown VSCode Distillation Bin PIP git Hotel Animate Github Statistics SQL v2ray CTC Color SPIE SQLite Pandas Freesound AI PDB QWEN Knowledge 搞笑 LLM NLTK 顶会 Cloudreve Pytorch CLAP Bert Qwen Safetensors COCO 多线程 LaTeX TensorFlow Food GGML hf TensorRT CC 净利润 Miniforge Video 版权 Paper Google tar OpenCV 财报 C++ Attention Web mmap Math ONNX Vmess Input Linux Datetime CSV WebCrawler SAM 阿里云 Search Magnet Interview XGBoost Rebuttal Quantize Tensor Firewall Crawler InvalidArgumentError NameSilo 继承 Streamlit Review 强化学习 PDF Git UNIX LoRA LeetCode Agent HuggingFace CUDA Hilton FP8 llama.cpp 云服务器 Dataset Tiktoken Ptyhon Excel Gemma Breakpoint Plotly VPN Ubuntu News Shortcut PyCharm 签证 Disk 论文速读 Sklearn Translation 关于博主 Nginx YOLO Use 腾讯云 Conda Pickle VGG-16 Michelin uWSGI DeepSeek
站点统计

本站现有博文334篇,共被浏览930802次

本站已经建立2639天!

热门文章
文章归档
回到顶部