# Hugging face LLM instead of OpenAI

**URL:** <https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432>\
**Category:** Rasa CALM\
**Created:** [March 27, 2024, 9:17am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432 "2024-03-27T09:17:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sekhar8](https://avatars.discourse-cdn.com/v4/letter/s/bc8723/32.png) [@sekhar8](https://forum.rasa.com/u/sekhar8)\
**Post date:** [March 27, 2024, 9:17am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/1 "2024-03-27T09:17:55Z")

</div>

Hi,

I’m new to rasa pro I want to use hugging face for the LLM instead of OpenAI.

I’m getting this error while running.

> C:\Users\chandrasekhar.m\AppData\Local\anaconda3\envs\plus\lib\site-packages\huggingface\_hub\utils\_deprecation.py:131: FutureWarning: ‘InferenceApi’ (from ‘huggingface\_hub.inference\_api’) is deprecated and will be removed from version ‘1.0’. `InferenceApi` client is deprecated in favor of the more feature-complete `InferenceClient`. Check out this guide to learn how to convert your script to use it: [Run Inference on servers](https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client). warnings.warn(warning\_message, FutureWarning) 2024-03-27 14:29:49 ERROR rasa.utils.log\_utils - [error] llm\_command\_generator.llm.error error=ValueError(‘Error raised by inference API: Request failed during generation: Server error: Out of available cache blocks: asked 52, only 11 free blocks’) C:\Users\chandrasekhar.m\AppData\Local\anaconda3\envs\plus\lib\site-packages\sanic\server\websockets\impl.py:521: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11. done, pending = await asyncio.wait( C:\Users\chandrasekhar.m\AppData\Local\anaconda3\envs\plus\lib\site-packages\huggingface\_hub\utils\_deprecation.py:131: FutureWarning: ‘InferenceApi’ (from ‘huggingface\_hub.inference\_api’) is deprecated and will be removed from version ‘1.0’. `InferenceApi` client is deprecated in favor of the more feature-complete `InferenceClient`. Check out this guide to learn how to convert your script to use it: [Run Inference on servers](https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client). warnings.warn(warning\_message, FutureWarning) 2024-03-27 14:31:56 ERROR rasa.utils.log\_utils - [error] llm\_command\_generator.llm.error error=ValueError(‘Error raised by inference API: Request failed during generation: Server error: Out of available cache blocks: asked 54, only 8 free blocks’) C:\Users\chandrasekhar.m\AppData\Local\anaconda3\envs\plus\lib\site-packages\sanic\server\websockets\impl.py:521: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11. done, pending = await asyncio.wait(

 ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/3X/9/2/92739f5134524bb917af3ef3a5f803df1250866c.png)

help me to understand how to use it.

---

<div class="post-metadata">

**Author:** ![Amina-Derouiche](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/amina-derouiche/32/23420_2.png) [@Amina-Derouiche](https://forum.rasa.com/u/Amina-Derouiche)\
**Post date:** [March 27, 2024, 10:09am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/2 "2024-03-27T10:09:53Z")

</div>

Hello @sekhar8,

Can you share with us the `config.yml` please ?

---

<div class="post-metadata">

**Author:** ![sekhar8](https://avatars.discourse-cdn.com/v4/letter/s/bc8723/32.png) [@sekhar8](https://forum.rasa.com/u/sekhar8)\
**Post date:** [March 27, 2024, 11:33am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/3 "2024-03-27T11:33:32Z")

</div>

sure,

```auto
recipe: default.v1
language: en
pipeline:
- name: LLMCommandGenerator
  llm:
    type: "huggingface_hub"
    repo_id: "HuggingFaceH4/zephyr-7b-beta"
    task: "text-generation"

  # model_name: gpt-4
policies:
- name: FlowPolicy
# - name: rasa_plus.ml.DocsearchPolicy
# - name: RulePolicy
assistant_id: 20240327-144556-amortized-beam

```

please suggest any good models for rephrasing the sentence also

---

<div class="post-metadata">

**Author:** ![abboudtest](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/abboudtest/32/24256_2.png) [@abboudtest](https://forum.rasa.com/u/abboudtest)\
**Post date:** [May 8, 2024, 9:11am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/4 "2024-05-08T09:11:58Z")

</div>

when I try it it gives me the following :

```auto
 Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors..

```

---

<div class="post-metadata">

**Author:** ![RakshitAralimatti](https://avatars.discourse-cdn.com/v4/letter/r/f6c823/32.png) [@RakshitAralimatti](https://forum.rasa.com/u/RakshitAralimatti)\
**Post date:** [May 9, 2024, 11:24am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/5 "2024-05-09T11:24:08Z")

</div>

Even I am getting same error even after using llama cpp after bot generating 2 or 3 responses. Can Any one please help?

---

<div class="post-metadata">

**Author:** ![abboudtest](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/abboudtest/32/24256_2.png) [@abboudtest](https://forum.rasa.com/u/abboudtest)\
**Post date:** [May 21, 2024, 11:55am UTC](https://forum.rasa.com/t/hugging-face-llm-instead-of-openai/61432/6 "2024-05-21T11:55:02Z")

</div>

I think that the LLM respond with a format that Rasa can’t understand it! So you may use a custom prompt template