I’ve been trying to replace the original DIETClassifier with the LLMIntentClassifier suggested by the official Rasa documentation, which can be found here: Using LLMs for Intent Classification. However, I’ve encountered several issues and am looking for solutions.
Issue 1: I followed the official documentation to modify my config.yml as follows:
recipe: default.v1
assistant_id: example_bot
language: zh
pipeline:
- name: LLMCommandGenerator
llm:
model_name: gpt-4
- name: test.CustomLLMIntentClassifier
fallback_intent: "out_of_scope"
llm:
model_name: gpt-4
temperature: 0.7
prompt: |
conversation with an intent. Reply ONLY with the name of the intent.
The intent should be one of the following:
- out_of_scope
- greet
- goodbye
- affirm
- regulation
- deny
- course_history
- ask_course
- inform
- ask_whatspossible
Message: Hello
Intent: greet
Message: Yes, I am
Intent: affirm
Message: hey there
Intent:
But when I run rasa train, I encounter an OSError: [Errno 36] File name too long. My current workaround is to use intent_classifier_template.jinja to wrap the prompt template, and I’ve modified config.yml as follows:
recipe: default.v1
assistant_id: example_bot
language: zh
pipeline:
- name: LLMCommandGenerator
llm:
model_name: gpt-4
- name: rasa_plus.ml.LLMIntentClassifier
fallback_intent: "out_of_scope"
llm:
model_name: gpt-4
temperature: 0.7
prompt: "./rasa_turtorial/intent_classifier_template.jinja"
policies:
- name: FlowPolicy
Is this approach viable?
Issue 2: After modifying, I ran rasa train and didn’t encounter any obvious issues. Next, I conducted testing with the following data:
version: "3.1"
nlu:
- intent: regulation
examples: |
- My student ID is used as the registration account, but I've forgotten the password, what should I do?
- My password is the last six digits of my ID card, but I am concerned about security issues. Can I change the password?
- intent: out_of_scope
examples: |
- Are you considering any internships lately? I heard XX company has opened a few good positions.
- What do you think is the hardest part of the internship interview process?
- intent: course_history
examples: |
- Record inquiry
- The history record just now
- intent: ask_course
examples: |
- Is there a python course offered in the Environmental Engineering Master's program this semester?
- Is there a history course offered at the Business School this semester?
However, when I run rasa test nlu, it seems that no classification is being performed at all. Do someone have any idea what might be the issue? I need help badly.