LLM_API_HEALTH_CHECK fails even though llm is available

Rasa CALM
1

Hi my first question on this forum: I use Rasa 3.14 and I want to use a self hosted llm from my machine. I use LMStudio to deploy it on localhost. But I get an error, that the LLM_API_HEALTH_CHECK fails. I couldnt find what that health check request looks like so I could test it manually. My LLM is reachable and supports the openai-like api.

config.yml
recipe: default.v1
language: de
assistant_id: stern-factory
pipeline:
- name: CompactLLMCommandGenerator
  llm:
    model_group: self_hosted_llm
  flow_retrieval:
    active: false
policies:
- name: FlowPolicy

endpoints.yml
# Allow rephrasing of responses using a Rasa-hosted model
nlg:
  type: rephrase
  llm:
    model_group: self_hosted_llm

model_groups:
  - id: self_hosted_llm  
    models:
      - provider: self-hosted
        model: openai/gpt-oss-20b
        api_base: "http://localhost:1234/v1"
  # - id: rasa_command_generation_model
    # models:
    #   - provider: rasa
    #     model: rasa/command-generator-llama-3.1-8b-instruct
    #     api_base: "https://tutorial-llm.rasa.ai"

I followed the steps from the documentation, but it doesn’t work. Because after setting LLM_API_HEALTH_CHECK to True. I get the following error:

2025-11-11 08:41:55 INFO     rasa.shared.utils.health_check.health_check  - [info     ] Sending a test LLM API request for the component - ContextualResponseRephraser. config={'model': 'openai/gpt-oss-20b', 'provider': 'self-hosted', 'api_base': 'http://localhost:1234/v1', 'api_version': None, 'api_type': 'openai', 'use_chat_completions_endpoint': True} event_key=contextual_response_rephraser.init.send_test_llm_api_request

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

2025-11-11 08:41:57 ERROR    rasa.cli.train  - [error    ] Test call to the LLM API failed for component - ContextualResponseRephraser. config={'model': 'openai/gpt-oss-20b', 'provider': 'self-hosted', 'api_base': 'http://localhost:1234/v1', 'api_version': None, 'api_type': 'openai', 'use_chat_completions_endpoint': True} error=ProviderClientAPIException('
Original error: litellm.APIError: APIError: Hosted_vllmException - Connection error.)') event_key=contextual_response_rephraser.init.send_test_llm_api_request_failed

Did anybody have the same problem?

2

Ok I solved it. For anyone who had the same problem. I followed the tutorial using a devcontainer. i forgot to use the docker variable for my host network.

Changing the config to:

     models:
       - provider: self-hosted
         model: openai/gpt-oss-20b
         api_base: "http://host.docker.internal:1234/v1"

did the trick