БЛОГ НА HUSL


ImportError: Using `load_in_8bit=True` requires Accelerate: но они у меня есть в программе pip freeze

Автор вопроса: Revolucion for Monica

Я пытаюсь запустить бэкенд gradio, но, похоже, мне не хватает библиотек, хотя они у меня загружены.

(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python3 gradio-chatbot.py 
/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/langchain/__init__.py:34: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
  warnings.warn(
Initializing backend for chatbot
/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/pipelines/__init__.py:698: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
  warnings.warn(
Traceback (most recent call last):
  File "/home/reply/Documents/chatbot-rag/gradio-chatbot.py", line 10, in <module>
    backend.load_embeddings_and_llm_models()
  File "/home/reply/Documents/chatbot-rag/backend.py", line 50, in load_embeddings_and_llm_models
    self.llm = self.load_llm(self.params)
  File "/home/reply/Documents/chatbot-rag/backend.py", line 66, in load_llm
    pipe = pipeline("text-generation", model=self.llm_name_or_path, model_kwargs=model_kwargs)
  File "/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/pipelines/__init__.py", line 870, in pipeline
    framework, model = infer_framework_load_model(
  File "/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
    model = model_class.from_pretrained(model, **kwargs)
  File "/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2714, in from_pretrained
    raise ImportError(
ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes`

Но они у меня есть. И они актуальны:

(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ pip freeze | grep -E '^(accelerate|bitsandbytes)'
accelerate==0.24.1
bitsandbytes==0.41.1
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ pip install --upgrade accelerate
pip install --upgrade -i https://test.pypi.org/simple/ bitsandbytes
Requirement already satisfied: accelerate in ./.venv/lib/python3.10/site-packages (0.24.1)
Requirement already satisfied: numpy>=1.17 in ./.venv/lib/python3.10/site-packages (from accelerate) (1.26.1)
Requirement already satisfied: packaging>=20.0 in ./.venv/lib/python3.10/site-packages (from accelerate) (23.2)
Requirement already satisfied: psutil in ./.venv/lib/python3.10/site-packages (from accelerate) (5.9.6)
Requirement already satisfied: pyyaml in ./.venv/lib/python3.10/site-packages (from accelerate) (6.0.1)
Requirement already satisfied: torch>=1.10.0 in ./.venv/lib/python3.10/site-packages (from accelerate) (2.1.0)
Requirement already satisfied: huggingface-hub in ./.venv/lib/python3.10/site-packages (from accelerate) (0.17.3)
Requirement already satisfied: filelock in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.13.1)
Requirement already satisfied: typing-extensions in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (4.8.0)
Requirement already satisfied: sympy in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (1.12)
Requirement already satisfied: networkx in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.2.1)
Requirement already satisfied: jinja2 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1.2)
Requirement already satisfied: fsspec in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2023.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.18.1 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.18.1)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: triton==2.1.0 in ./.venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.1.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in ./.venv/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.10.0->accelerate) (12.3.52)
Requirement already satisfied: requests in ./.venv/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in ./.venv/lib/python3.10/site-packages (from huggingface-hub->accelerate) (4.66.1)
Requirement already satisfied: MarkupSafe>=2.0 in ./.venv/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.venv/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venv/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2023.7.22)
Requirement already satisfied: mpmath>=0.19 in ./.venv/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)
Looking in indexes: https://test.pypi.org/simple/
Requirement already satisfied: bitsandbytes in ./.venv/lib/python3.10/site-packages (0.41.1)

Интерпретатором является virtualenv:

(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ which python
/home/reply/Documents/chatbot-rag/.venv/bin/python
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ which python3
/home/reply/Documents/chatbot-rag/.venv/bin/python3

У меня установлена библиотека TORCH:

(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ pip uninstall transformers torch
Found existing installation: transformers 4.35.0
Uninstalling transformers-4.35.0:
  Would remove:
    /home/reply/Documents/chatbot-rag/.venv/bin/transformers-cli
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers-4.35.0.dist-info/*
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/transformers/*
Proceed (Y/n)? n
Found existing installation: torch 2.1.0
Uninstalling torch-2.1.0:
  Would remove:
    /home/reply/Documents/chatbot-rag/.venv/bin/convert-caffe2-to-onnx
    /home/reply/Documents/chatbot-rag/.venv/bin/convert-onnx-to-caffe2
    /home/reply/Documents/chatbot-rag/.venv/bin/torchrun
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/functorch/*
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/nvfuser/*
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/torch-2.1.0.dist-info/*
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/torch/*
    /home/reply/Documents/chatbot-rag/.venv/lib/python3.10/site-packages/torchgen/*

Но он не может ее найти:

(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ pip env
ERROR: unknown command "env"
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ.get("USE_TORCH")
>>> 
(.venv) reply@reply-GP66-Leopard-11UH:~/Documents/chatbot-rag$ python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ.get("USE_TORCH")
>>>

Источник

Ответы (0 шт):

licensed under cc by-sa 3.0 with attribution.