EADST

Sharding and SafeTensors in Hugging Face Transformers

In the Hugging Face transformers library, managing large models efficiently is crucial, especially when working with limited disk space or specific file size requirements. Two key features that help with this are sharding and the use of SafeTensors.

Sharding

Sharding is the process of splitting a large model's weights into smaller files or "shards." This is particularly useful when dealing with large models that exceed file size limits or when you want to manage storage more effectively.

Usage

To shard a model during the saving process, you can use the max_shard_size parameter in the save_pretrained method. Here's an example:

# Save the model with sharding, setting the maximum shard size to 1GB
model.save_pretrained('./model_directory', max_shard_size="1GB")

In this example, the model's weights will be divided into multiple files, each not exceeding 1GB. This can make storage and transfer more manageable, especially when dealing with large-scale models.

SafeTensors

The safetensors library provides a new format for storing tensors in a safe and efficient way. Unlike traditional formats like PyTorch's .pt files, SafeTensors ensures that the tensor data cannot be accidentally executed as code, offering an additional layer of security. This is particularly important when sharing models across different systems or with the community.

Usage

To save a model using SafeTensors, simply specify the safe_serialization parameter when saving:

# Save the model using SafeTensors format
model.save_pretrained('./model_directory', safe_serialization=True)

This will create files with the .safetensors extension, ensuring the saved tensors are stored safely.

Combining Sharding and SafeTensors

You can combine both sharding and SafeTensors to save a large model securely and efficiently:

# Save the model with sharding and SafeTensors
model.save_pretrained('./model_directory', max_shard_size="1GB", safe_serialization=True)

This setup splits the model into shards, each in the SafeTensors format, offering both manageability and security.

Conclusion

By leveraging sharding and SafeTensors, Hugging Face transformers users can handle large models more effectively. Sharding helps manage file sizes, while SafeTensors ensures the safe storage of tensor data. These features are essential for anyone working with large-scale models, providing both practical and security benefits.

相关标签
About Me
XD
Goals determine what you are going to be.
Category
标签云
Diagram Transformers 关于博主 报税 Python ModelScope Hungarian SQL Miniforge 继承 API Streamlit Base64 YOLO Windows uWSGI CEIR Password LeetCode 长期记忆 IndexTTS2 Plate MD5 FlashAttention CSV Hotel OpenAI LLAMA Website PyTorch Docker CV Color 证件照 TensorFlow WAN 递归学习法 Heatmap Cloudreve Freesound Statistics Anaconda 第一性原理 Distillation Harness FP32 Food 论文 Tensor 阿里云 Nginx EXCEL SAM UNIX Disk SVR API网关 GPTQ llama.cpp Jupyter 域名 净利润 icon NLTK QWEN hf NameSilo Gemma CUDA scipy XGBoost GoogLeNet ChatGPT Mixtral Pickle Review XML v0.dev torchinfo Baidu News Data BTC Llama PIP transformers tqdm Claude PDF Logo LaTeX Tiktoken printf Qwen2.5 Ptyhon mmap InvalidArgumentError Land BF16 Random 音频 Input HaggingFace Bert Math WebCrawler CC Sklearn ResNet-50 Bin 版权 Markdown Vmess Qwen GPT4 git-lfs Datetime Shortcut Pillow Google Dataset 腾讯云 图标 FP64 Safetensors Breakpoint OCR Michelin LLM Knowledge Pandas TensorRT Linux Use v2ray Bipartite Firewall Zip Jev Github HuggingFace Tracking Interview Conda Proxy Clash logger Plotly Web Rebuttal Pytorch BeautifulSoup DeepStream Magnet Paper Card VPN OpenCV Augmentation Attention FP16 GIT CTC ONNX ms-swift 多进程 Image2Text diffusers Crawler Git COCO C++ Jetson 图形思考法 tar 云服务器 Vim Domain DeepSeek JSON SPIE Algorithm Hilton FastAPI Ubuntu Numpy Video Django Template 飞书 AI 签证 Translation TSV Excel 多线程 Bitcoin Paddle GGML Quantize NLP 算法题 Agent SQLite Animate git 论文速读 PyCharm RGB uwsgi TTS PDB Qwen2 VGG-16 搞笑 财报 公式 CAM RL Quantization Search 自我改进 个人博客 顶会 RAR 强化学习 CLAP Permission LoRA VSCode FP8 UI
站点统计

本站现有博文340篇,共被浏览1015455次

本站已经建立2717天!

热门文章
文章归档
回到顶部