# Hugging Face LLM ‘huggingface\_hub.inference\_api’ is deprecated,

**URL:** <https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829>\
**Category:** Rasa CALM\
**Created:** [May 2, 2024, 7:09am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829 "2024-05-02T07:09:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sharen\_Etiqa](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sharen\_Etiqa](https://forum.rasa.com/u/Sharen_Etiqa)\
**Post date:** [May 2, 2024, 7:09am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829/1 "2024-05-02T07:09:26Z")

</div>

Hi! I’m currently using Rasa Pro Developer’s Edition with CALM. I’m trying to load hugging face models with `LLMCommandGenerator`. However, it seems like Rasa Pro is using `huggingface_hub.inference_api` or `InferenceApi` which is deprecated. The new one is `InferenceClient`. Is there any way to fix this? Thank you!

---

<div class="post-metadata">

**Author:** ![l2nguyen](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/l2nguyen/32/23881_2.png) [@l2nguyen](https://forum.rasa.com/u/l2nguyen)\
**Post date:** [May 2, 2024, 8:13am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829/2 "2024-05-02T08:13:59Z")

</div>

short answer is rasa pro would need to upgrade the langchain version it’s using. However, according to [this](https://github.com/langchain-ai/langchain/issues/10483#issuecomment-1867098864) the inferenceApi should still work even though it’s deprecated. Are you running into issues using hugging face models?

Rasa Pro uses `langchain` to integrate with different model providers. It looks like this change from `InferenceApi` to `InferenceClient` was included in [langchain v0.1.4](https://github.com/langchain-ai/langchain/releases/tag/v0.1.4) released in Jan 2024 - it’s this [PR](https://github.com/langchain-ai/langchain/pull/14406).

---

<div class="post-metadata">

**Author:** ![Sharen\_Etiqa](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sharen\_Etiqa](https://forum.rasa.com/u/Sharen_Etiqa)\
**Post date:** [May 2, 2024, 8:49am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829/3 "2024-05-02T08:49:33Z")

</div>

A part of my config file looks like this:

```auto
...
- name: EnterpriseSearchPolicy
  llm: 
     type: "huggingface_hub"
...

```

I’m trying a build a simple RAG application using the Mixtral open-sourced model from Hugging Face. However, I’m getting the error mentioned above. Do I need to change anything from here? Previously, I’ve built RAG applications with langchain and Hugging Face models and I didn’t encounter any issues. I’m having this problem only with Rasa Pro.

---

<div class="post-metadata">

**Author:** ![l2nguyen](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/l2nguyen/32/23881_2.png) [@l2nguyen](https://forum.rasa.com/u/l2nguyen)\
**Post date:** [May 3, 2024, 9:19am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829/4 "2024-05-03T09:19:54Z")

</div>

It would be helpful if you shared your full config and the error message that you’re seeing in rasa. From what I understand, the deprecation is a warning so it should still work.

For `EnterpriseSearchPolicy`, you would need to set the config for an embedding and llm model too. see [here](https://rasa.com/docs/rasa-pro/concepts/policies/enterprise-search-policy#llm--embeddings)

You would also need to set the `LLMCommandGenerator` to use models from hugging face. In 3.8, the addition of the [flow retrieval](https://rasa.com/docs/rasa-pro/concepts/dialogue-understanding/#customizing-flow-retrieval) feature means you need to set an embedding model too if you want to use that feature.

I haven’t tried using hugging face with rasa pro yet. The docs for using hugging face models are [here](https://rasa.com/docs/rasa-pro/concepts/components/llm-configuration/#hugging-face-hub) for llm models. and [here](https://rasa.com/docs/rasa-pro/concepts/components/llm-configuration#hugging-face-hub-1) for embedding models but it looks similar for the llm config.

---

<div class="post-metadata">

**Author:** ![Sharen\_Etiqa](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sharen\_Etiqa](https://forum.rasa.com/u/Sharen_Etiqa)\
**Post date:** [May 6, 2024, 12:23am UTC](https://forum.rasa.com/t/hugging-face-llm-huggingface-hub-inference-api-is-deprecated/61829/5 "2024-05-06T00:23:19Z")

</div>

Hi. Thanks for the information. Here’s my config file:

```auto
pipeline:
  - name: LLMCommandGenerator
       llm: 
          type: "huggingface_hub"
          repo_id: "mistralai/Mixtral-8x7B-Instruct-v0.1"
       embeddings:
           type: "huggingface_hub"
           repo_id: "sentence-transformers/all-mpnet-base-v2"

policies:
- name: EnterpriseSearchPolicy
  llm: 
     type: "huggingface_hub"
     repo_id: "mistralai/Mixtral-8x7B-Instruct-v0.1"
  embeddings:
    type: "huggingface_hub"
    repo_id: "sentence-transformers/all-mpnet-base-v2"
  vector_store:
       type: "faiss"
       source: "./docs"
  citation_enabled: True

```

Here’s the error:

```auto
FutureWarning: 'InferenceApi (from 'huggingface_hub.inference_api') is deprecated and will be removed from version '1.0''. 'InferenceApi' client is deprecated in favor of the more feature-complete 'InferenceClient'. 

```

Please correct me if I’m not using it correctly. Thank you.