Entity not recogised properly

Rasa Open Source
1

Testing_Data_Copy.json (113.8 KB)

Hi I have uploaded the training data json file for the reference .

My Question for that chatbot that i made is “What is the average ratings of movies produced by Karan JOhar in last 10 years ?”

In this case “Karan Johar” should be recognizd as “producer” entity .

In many cases i have marked “Karan JOhar” as "director " entity.

But when a phrase " produced by " or "director by " comes, in those case the name after the phrase should belong to “producer”/“director” depending on the phrase preffixed.

Below is the result that i got from my model.

{‘intent’: {‘name’: ‘rating_search’, ‘confidence’: 0.9789842971636689}, ‘entities’: [{‘start’: 12, ‘end’: 27, ‘value’: ‘average ratings’, ‘entity’: ‘aggmethod’, ‘confidence’: 0.9984705847092747, ‘extractor’: ‘CRFEntityExtractor’}, {‘start’: 38, ‘end’: 49, ‘value’: ‘produced by’, ‘entity’: ‘role’, ‘confidence’: 0.970444154132258, ‘extractor’: ‘CRFEntityExtractor’}, {‘start’: 50, ‘end’: 61, ‘value’: ’ karan johar’, ‘entity’: ‘director’ , ‘confidence’: 0.9519798163153366, ‘extractor’: ‘CRFEntityExtractor’}, {‘start’: 62, ‘end’: 78, ‘value’: ‘in last 10 years’, ‘entity’: ‘time’, ‘confidence’: 0.8693345972879974, ‘extractor’: ‘CRFEntityExtractor’}], ‘intent_ranking’: [{‘name’: ‘rating_search’, ‘confidence’: 0.9789842971636689}, {‘name’: ‘movie_search’, ‘confidence’: 0.011960894645934217}, {‘name’: ‘actor_search’, ‘confidence’: 0.0029286358141673865}, {‘name’: ‘producer_search’, ‘confidence’: 0.0018446370991670315}, {‘name’: ‘director_search’, ‘confidence’: 0.001321095084823971}, {‘name’: ‘coworker_search’, ‘confidence’: 0.0009156458042880051}, {‘name’: ‘affirm’, ‘confidence’: 0.0007931370517594824}, {‘name’: ‘actress_search’, ‘confidence’: 0.000524613144352204}, {‘name’: ‘goodbye’, ‘confidence’: 0.0004869859850510281}, {‘name’: ‘greet’, ‘confidence’: 0.0002400582067874318}], ‘text’: ‘What is the average ratings of movies produced by Karan JOhar in last 10 years ?’}

If you c , Karan Johar is marked as director entity inspite of the phrase “produced by” preffixing it. So my question is what does the model learn then ?

And if you see the result, there are other intents rankings . Similarly how to check the other for entities rankings ? Bcz i believe that entities are picked upon based on the entity confidence.

Does my model doesnt recognize Karan Johar as a producer because there is not enough training sample cases as him pointing to a producer?

2

What pipeline are you using?

Did you already test your NLU model as described here? If you test your model, you will get a detailed report. It shows you the accuracy of the producer entity, which might help to figure out what exactly is going on.

3

Ya i tested and got the ans . Its working fine.

I have a question .

Like how we get the intents rankings of other intents for the question parsed, can we get the entities rankings of other entities as well ??

4

What was the problem?

As far as I know you cannot obtain the entity rankings right now. But might be a good addition. Will open a GitHub issue for that so that we can discuss if/how we are going to implement it.

5

Thanks Tanja.

Actually my issue right now is , im training a model and passing question to the model to which it would return a dict of results .

PLZ FIND THE CODE BELOW FOR THE SAME :

def run_nlu(): interpreter = Interpreter.load(’./Rasa/models/nlu-20190622-162513/nlu/’) question = input ("QUESTION FOR CHATBOT ") model_result = interpreter.parse(question)
return model_result

In the model results, i could find the intent that the model recognised and also the other intents ranking in that result like below :

‘intent_ranking’: [{‘name’: ‘movie_search’, ‘confidence’: 0.9998579472281128}, {‘name’: ‘rating_search’, ‘confidence’: 0.00014184487052934283}, {‘name’: ‘producer_search’, ‘confidence’: 1.60321452900132e-07}, {‘name’: ‘actor_search’, ‘confidence’: 1.4431655511242704e-08}, {‘name’: ‘director_search’, ‘confidence’: 1.006682290416047e-08}, {‘name’: ‘costar_search’, ‘confidence’: 6.847406360245541e-09}, {‘name’: ‘actress_search’, ‘confidence’: 4.857948860319296e-09}, {‘name’: ‘affirm’, ‘confidence’: 4.229151238816665e-09}, {‘name’: ‘goodbye’, ‘confidence’: 3.990133422750423e-09}, {‘name’: ‘greet’, ‘confidence’: 3.1567867160003953e-09}], ‘text’: ‘Top 5 movies directed by Kalpit Deshai in last 5 years ?’}

It also predicts the entities of the question parsed.

My doubt is can it give all the entities ranking for each and every single entity that it finds like how it gave results for intent rankings and from which it chooses the one with highest ranking ?

Are there any ways to get them ??

6

@Aswinprabhakaran Unfortunately, you cannot get them right now. Feel free to add your use case to Entity ranking · Issue #3854 · RasaHQ/rasa · GitHub.