Hugging Face LLM ‘huggingface_hub.inference_api’ is deprecated,

Rasa Pro CALM
1

Hi! I’m currently using Rasa Pro Developer’s Edition with CALM. I’m trying to load hugging face models with LLMCommandGenerator. However, it seems like Rasa Pro is using huggingface_hub.inference_api or InferenceApi which is deprecated. The new one is InferenceClient. Is there any way to fix this? Thank you!

2

short answer is rasa pro would need to upgrade the langchain version it’s using. However, according to this the inferenceApi should still work even though it’s deprecated. Are you running into issues using hugging face models?

Rasa Pro uses langchain to integrate with different model providers. It looks like this change from InferenceApi to InferenceClient was included in langchain v0.1.4 released in Jan 2024 - it’s this PR.

3

A part of my config file looks like this:

...
- name: EnterpriseSearchPolicy
  llm: 
     type: "huggingface_hub"
...

I’m trying a build a simple RAG application using the Mixtral open-sourced model from Hugging Face. However, I’m getting the error mentioned above. Do I need to change anything from here? Previously, I’ve built RAG applications with langchain and Hugging Face models and I didn’t encounter any issues. I’m having this problem only with Rasa Pro.

4

It would be helpful if you shared your full config and the error message that you’re seeing in rasa. From what I understand, the deprecation is a warning so it should still work.

For EnterpriseSearchPolicy, you would need to set the config for an embedding and llm model too. see here

You would also need to set the LLMCommandGenerator to use models from hugging face. In 3.8, the addition of the flow retrieval feature means you need to set an embedding model too if you want to use that feature.

I haven’t tried using hugging face with rasa pro yet. The docs for using hugging face models are here for llm models. and here for embedding models but it looks similar for the llm config.

5

Hi. Thanks for the information. Here’s my config file:

pipeline:
  - name: LLMCommandGenerator
       llm: 
          type: "huggingface_hub"
          repo_id: "mistralai/Mixtral-8x7B-Instruct-v0.1"
       embeddings:
           type: "huggingface_hub"
           repo_id: "sentence-transformers/all-mpnet-base-v2"

policies:
- name: EnterpriseSearchPolicy
  llm: 
     type: "huggingface_hub"
     repo_id: "mistralai/Mixtral-8x7B-Instruct-v0.1"
  embeddings:
    type: "huggingface_hub"
    repo_id: "sentence-transformers/all-mpnet-base-v2"
  vector_store:
       type: "faiss"
       source: "./docs"
  citation_enabled: True

Here’s the error:

FutureWarning: 'InferenceApi (from 'huggingface_hub.inference_api') is deprecated and will be removed from version '1.0''. 'InferenceApi' client is deprecated in favor of the more feature-complete 'InferenceClient'.

Please correct me if I’m not using it correctly. Thank you.