# How to use Ollama models in Rasa CALM?

**URL:** <https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823>\
**Category:** Rasa CALM\
**Created:** [May 1, 2024, 3:22pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823 "2024-05-01T15:22:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![vishnupriyavr](https://avatars.discourse-cdn.com/v4/letter/v/a3d4f5/32.png) [@vishnupriyavr](https://forum.rasa.com/u/vishnupriyavr)\
**Post date:** [May 1, 2024, 3:22pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/1 "2024-05-01T15:22:18Z")

</div>

Hello, I am trying to use models served through Ollama into Rasa CALM? I could not find it anywhere in the documentation, any leads or tips would be helpful

---

<div class="post-metadata">

**Author:** ![camattin](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/camattin/32/18990_2.png) [@camattin](https://forum.rasa.com/u/camattin)\
**Post date:** [May 1, 2024, 4:22pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/2 "2024-05-01T16:22:37Z")

</div>

Hi Vishni,

The easiest way I’ve found to use Ollama (presuming you’re running the Ollama server locally) is to make use of its OpenAI-compatible entry point.

Your config would look similar to:

```auto
pipeline:
- name: LLMCommandGenerator
  llm:
    model: "wizardlm2:7b"
    max_tokens: 20
    type: openai
    openai_api_base: http://localhost:11434/v1
    openai_api_key: foobar

```

And, of course, you can replace the value for `model` with whichever model you have downloaded to your Ollama installation.

Hope this helps!

---

<div class="post-metadata">

**Author:** ![vishnupriyavr](https://avatars.discourse-cdn.com/v4/letter/v/a3d4f5/32.png) [@vishnupriyavr](https://forum.rasa.com/u/vishnupriyavr)\
**Post date:** [May 2, 2024, 3:26am UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/3 "2024-05-02T03:26:13Z")

</div>

Hi Chris, Thanks for the quick response! Sure, this is of great help, will try it out.

---

<div class="post-metadata">

**Author:** ![MannavaVivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/mannavavivek/32/24083_2.png) [@MannavaVivek](https://forum.rasa.com/u/MannavaVivek)\
**Post date:** [May 13, 2024, 3:36pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/4 "2024-05-13T15:36:14Z")

</div>

```auto
2024-05-13 17:28:19 WARNING langchain.llms.base - Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 8.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=7).
2024-05-13 17:28:34 WARNING langchain.llms.base - Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 10.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=7).
2024-05-13 17:28:51 ERROR rasa.utils.log_utils - [error] llm_command_generator.llm.error error=Timeout(message="Request timed out: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=7)", http_status=None, request_id=None)

```

I tried to use Ollama as instructed, and it threw me this error. I suspect its the same issue discussed [here](https://github.com/langchain-ai/langchain/issues/3512) and according to the status of the issue, doesn’t seem to be getting a fix anytime soon. So can anyone suggest alternatives for running local models?

---

<div class="post-metadata">

**Author:** ![camattin](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/camattin/32/18990_2.png) [@camattin](https://forum.rasa.com/u/camattin)\
**Post date:** [May 13, 2024, 6:03pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/5 "2024-05-13T18:03:18Z")

</div>

Hi Vivek,

Are you able to access your locally running Ollama server with curl? e.g.

`curl http://localhost:11434/api/generate -d '{ "model": "phi3", "prompt": "Why is the sky blue?", "stream": false }' `

---

<div class="post-metadata">

**Author:** ![MannavaVivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/mannavavivek/32/24083_2.png) [@MannavaVivek](https://forum.rasa.com/u/MannavaVivek)\
**Post date:** [May 14, 2024, 8:31am UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/6 "2024-05-14T08:31:50Z")

</div>

Yes, Ollama is accessible via curl and the python openAI api interface, but doesn’t seem to work when called via Rasa. The config is

```auto
pipeline:
- name: LLMCommandGenerator
  llm:
    model: "llama3:latest"
    type: openai
    openai_api_base: http://localhost:11434/v1
    openai_api_key: foobar

```

---

<div class="post-metadata">

**Author:** ![Rabi](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/rabi/32/12756_2.png) [@Rabi](https://forum.rasa.com/u/Rabi)\
**Post date:** [September 24, 2024, 1:36pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/7 "2024-09-24T13:36:46Z")

</div>

@MannavaVivek did you found any solution?

---

<div class="post-metadata">

**Author:** ![MannavaVivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/mannavavivek/32/24083_2.png) [@MannavaVivek](https://forum.rasa.com/u/MannavaVivek)\
**Post date:** [September 24, 2024, 3:55pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/8 "2024-09-24T15:55:44Z")

</div>

With the upgraded version of Rasa(mine is 3.9.3), you can directly call the model

> type: ollama model: “llama3.1:8b”

---

<div class="post-metadata">

**Author:** ![pkchoudhary1211](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/pkchoudhary1211/32/5790_2.png) [@pkchoudhary1211](https://forum.rasa.com/u/pkchoudhary1211)\
**Post date:** [September 27, 2024, 3:23pm UTC](https://forum.rasa.com/t/how-to-use-ollama-models-in-rasa-calm/61823/9 "2024-09-27T15:23:01Z")

</div>

Did the update solve it?