mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-20 04:58:22 +06:00
28de2f4de3
* start integration * fix * add and debug tests * update tests * make pytorch serialization works * compatible with device_map and offload * fix tests * make style * add ref * guard against safetensors * add float8 and style * fix is_serializable * Fix shard_checkpoint compatibility with quanto * more tests * docs * adjust memory * better * style * pass tests * Update src/transformers/modeling_utils.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * add is_safe_serialization instead * Update src/transformers/quantizers/quantizer_quanto.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * add QbitsTensor tests * fix tests * simplify activation list * Update docs/source/en/quantization.md Co-authored-by: David Corvoysier <david.corvoysier@gmail.com> * better comment * Update tests/quantization/quanto_integration/test_quanto.py Co-authored-by: David Corvoysier <david.corvoysier@gmail.com> * Update tests/quantization/quanto_integration/test_quanto.py Co-authored-by: David Corvoysier <david.corvoysier@gmail.com> * find and fix edge case * Update docs/source/en/quantization.md Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * pass weights_only_kwarg instead * fix shard_checkpoint loading * simplify update_missing_keys * Update tests/quantization/quanto_integration/test_quanto.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * recursion to get all tensors * block serialization * skip serialization tests * fix * change by cuda:0 for now * fix regression * update device_map * fix doc * add noteboon * update torch_dtype * update doc * typo * typo * remove comm --------- Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> Co-authored-by: David Corvoysier <david.corvoysier@gmail.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> Co-authored-by: Younes Belkada <younesbelkada@gmail.com>
53 lines
2.4 KiB
Docker
53 lines
2.4 KiB
Docker
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
|
|
LABEL maintainer="Hugging Face"
|
|
|
|
ARG DEBIAN_FRONTEND=noninteractive
|
|
|
|
# Use login shell to read variables from `~/.profile` (to pass dynamic created variables between RUN commands)
|
|
SHELL ["sh", "-lc"]
|
|
|
|
# The following `ARG` are mainly used to specify the versions explicitly & directly in this docker file, and not meant
|
|
# to be used as arguments for docker build (so far).
|
|
|
|
ARG PYTORCH='2.2.0'
|
|
# Example: `cu102`, `cu113`, etc.
|
|
ARG CUDA='cu118'
|
|
|
|
RUN apt update
|
|
RUN apt install -y git libsndfile1-dev tesseract-ocr espeak-ng python python3-pip ffmpeg
|
|
RUN python3 -m pip install --no-cache-dir --upgrade pip
|
|
|
|
ARG REF=main
|
|
RUN git clone https://github.com/huggingface/transformers && cd transformers && git checkout $REF
|
|
|
|
RUN [ ${#PYTORCH} -gt 0 ] && VERSION='torch=='$PYTORCH'.*' || VERSION='torch'; echo "export VERSION='$VERSION'" >> ~/.profile
|
|
RUN echo torch=$VERSION
|
|
# `torchvision` and `torchaudio` should be installed along with `torch`, especially for nightly build.
|
|
# Currently, let's just use their latest releases (when `torch` is installed with a release version)
|
|
RUN python3 -m pip install --no-cache-dir -U $VERSION torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/$CUDA
|
|
|
|
RUN python3 -m pip install --no-cache-dir -e ./transformers[dev-torch]
|
|
|
|
RUN python3 -m pip install --no-cache-dir git+https://github.com/huggingface/accelerate@main#egg=accelerate
|
|
|
|
# Add bitsandbytes for mixed int8 testing
|
|
RUN python3 -m pip install --no-cache-dir bitsandbytes
|
|
|
|
# Add auto-gptq for gtpq quantization testing
|
|
RUN python3 -m pip install --no-cache-dir auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/
|
|
|
|
# Add optimum for gptq quantization testing
|
|
RUN python3 -m pip install --no-cache-dir git+https://github.com/huggingface/optimum@main#egg=optimum
|
|
|
|
# Add aqlm for quantization testing
|
|
RUN python3 -m pip install --no-cache-dir aqlm[gpu]==1.0.2
|
|
|
|
# Add autoawq for quantization testing
|
|
RUN python3 -m pip install --no-cache-dir https://github.com/casper-hansen/AutoAWQ/releases/download/v0.2.0/autoawq-0.2.0+cu118-cp38-cp38-linux_x86_64.whl
|
|
|
|
# Add quanto for quantization testing
|
|
RUN python3 -m pip install --no-cache-dir quanto
|
|
|
|
# When installing in editable mode, `transformers` is not recognized as a package.
|
|
# this line must be added in order for python to be aware of transformers.
|
|
RUN cd transformers && python3 setup.py develop |