Intent is not being identified

Hi,

I have an intent called enter_data that has some examples that are plates of vehicles. They are composed of 3 letters and 4 numbers. Here is the example list:

## intent:enter_data
- DFCUK13HB108904
- DBCAN17JB222213
- 074.768.329-92
- +5541988080412
- 80730-000
- [Por telefone](contact_method)
- [Via chat](contact_method)
- BAB-5293
- QIG-0530
- CDE0294
- FGH1456
- qig0530
- dfe4901
- bbh2247
- NAI-2502
- HON-9500
- HZT-5576
- BIZ-2802
- NAZ-0017
- LVM-0035
- KFN-8264
- LDB-6557
- HZT-6912
- LWB-6765
- NAG-6719
- MEA-1826
- MXC-5037
- JFK-2497
- AYC-8950
- ATC-9693
- DZD-2581
- EEQ-7761
- DKT-3566
- BVW1882
- FLJ8783
- CET0111
- GHH3809
- GDW6475
- FST2523
- EQL9528
- DML1396
- BZS1696
- flk2308
- bsz4104
- pqi9356
- qin6490
- okx8563
- axt5782
- gab7287
- ppz5850
- giq9538
- fkg0962
- kyy-7457
- htn-7749
- gbx-2862
- lmh-6087
- oar-1983
- gjo-3165
- bco-4392

When i type something like: bab-5293 cde-9312 ACD-2812 EXI-1029

Rasa always identifies the intent correctly as enter_data. But when i type without the dash (-) for example: acd1249 DFG9182 PEO1234

It never identifies the intent enter_data it always return Intent: None and because of that the fallback is always called. How can I improve the identification for that intent? Because there are a lot of examples without dash already and the performance has not increase. Should I just add more examples? Or there is a better approach for this problem?

PS.: I use that in a form action. And although the form runs correctly because of the Intent None in the end of the form it shouts the fallback.

The messages returned are:

2019-01-07 12:27:43 DEBUG    rasa_core.processor  - Received user message 'bab5293' with intent '{'name': None, 'confidence': 0.0}' and entities '[{'start': 3, 'end': 7, 'text': '5293', 'value': 5293, 'confidence': 1.0, 'additional_info': {'value': 5293, 'type': 'value'}, 'entity': 'number', 'extractor': 'ner_duckling_http'}]'
2019-01-07 12:27:43 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 16 events
2019-01-07 12:27:43 DEBUG    rasa_core.processor  - Current slot values: 
        car_chassi: None
        car_model: None
        car_plate: None
        car_version: None
        contact_method: None
        dealership_zip: None
        requested_slot: car_plate
        retry_question: None
        user_cpf: None
        user_email: None
        user_phone: None
        user_zip: None
2019-01-07 12:27:43 DEBUG    rasa_core.policies.memoization  - Current tracker state [{}, {'prev_action_listen': 1.0, 'intent_greet': 1.0}, {'intent_greet': 1.0, 'prev_utter_greet': 1.0}, {'prev_action_menu': 1.0, 'intent_greet': 1.0}, {'prev_action_listen': 1.0, 'intent_repairs': 1.0}]
2019-01-07 12:27:43 DEBUG    rasa_core.policies.memoization  - There is a memorised next action '45'
2019-01-07 12:27:43 DEBUG    rasa_core.policies.fallback  - NLU confidence 0.0 is lower than NLU threshold 0.4. Predicting fallback action: action_default_fallback
2019-01-07 12:27:43 DEBUG    rasa_core.policies.form_policy  - There is an active form 'repair_form'
2019-01-07 12:27:43 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_3_FormPolicy
2019-01-07 12:27:43 DEBUG    rasa_core.processor  - Predicted next action 'repair_form' with prob 1.00.
2019-01-07 12:27:43 DEBUG    rasa_core.actions.action  - Calling action endpoint to run action 'repair_form'.
DEBUG:rasa_core_sdk.executor:Received request to run 'repair_form'
DEBUG:rasa_core_sdk.forms:The form '{'name': 'repair_form', 'validate': True, 'rejected': False}' is active
DEBUG:rasa_core_sdk.forms:Validating user input '{'intent': {'name': None, 'confidence': 0.0}, 'entities': [{'start': 3, 'end': 7, 'text': '5293', 'value': 5293, 'confidence': 1.0, 'additional_info': {'value': 5293, 'type': 'value'}, 'entity': 'number', 'extractor': 'ner_duckling_http'}], 'intent_ranking': [], 'text': 'bab5293'}'
DEBUG:rasa_core_sdk.forms:Trying to extract requested slot 'car_plate' ...
DEBUG:rasa_core_sdk.forms:Got mapping '{'type': 'from_text', 'intent': [], 'not_intent': []}'
DEBUG:rasa_core_sdk.forms:Successfully extracted 'bab5293' for requested slot 'car_plate'
DEBUG:actions:No slots left to request
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 192.168.0.200
DEBUG:urllib3.connectionpool:http://192.168.0.200:7071 "GET /index.php?plate=bab5293 HTTP/1.1" 200 282
{'codigoRetorno': '0', 'mensagemRetorno': 'Sem erros.', 'codigoSituacao': '0', 'situacao': 'Sem restrição', 'modelo': 'HONDA/FIT LX CVT', 'marca': 'HONDA/FIT LX CVT', 'cor': 'Cinza', 'ano': '2015', 'anoModelo': '2016', 'placa': 'BAB5293', 'data': '07/01/2019 às 12:27:44', 'uf': 'PR', 'municipio': 'CURITIBA', 'chassi': '17742', 'dataAtualizacaoCaracteristicasVeiculo': '09/11/2018', 'dataAtualizacaoRouboFurto': '06/01/2019', 'dataAtualizacaoAlarme': '06/01/2019'}
DEBUG:rasa_core_sdk.forms:Deactivating the form 'repair_form'
DEBUG:rasa_core_sdk.executor:Successfully ran 'repair_form'
127.0.0.1 - - [2019-01-07 12:27:44] "POST /webhook HTTP/1.1" 200 581 0.773670
2019-01-07 12:27:44 INFO     rasa_core.channels.facebook  - Sending message: Aguarde enquanto estou pesquisando...
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Action 'repair_form' ended with events '['SlotSet(key: car_plate, value: bab5293)', 'SlotSet(key: retry_question, value: None)', 'SlotSet(key: car_plate, value: None)', 'Form(None)', 'SlotSet(key: requested_slot, value: None)']'
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: Aguarde enquanto estou pesquisando..., data: {
  "elements": null,
  "buttons": null,
  "attachment": null
})'
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: Vejo que seu carro ainda não é um Nissan! Deseja marcar um test drive?, data: {
  "elements": null,
  "buttons": [
    {
      "payload": "Test drive",
      "title": "Test drive",
      "type": "postback"
    },
    {
      "payload": "Voltar ao menu",
      "title": "Voltar ao menu",
      "type": "postback"
    }
  ],
  "attachment": null
})'
2019-01-07 12:27:45 DEBUG    rasa_core.policies.memoization  - Current tracker state [{'prev_action_listen': 1.0, 'intent_greet': 1.0}, {'intent_greet': 1.0, 'prev_utter_greet': 1.0}, {'prev_action_menu': 1.0, 'intent_greet': 1.0}, {'prev_action_listen': 1.0, 'intent_repairs': 1.0}, {'prev_repair_form': 1.0, 'intent_repairs': 1.0}]
2019-01-07 12:27:45 DEBUG    rasa_core.policies.memoization  - There is a memorised next action '0'
2019-01-07 12:27:45 DEBUG    rasa_core.policies.fallback  - NLU confidence 0.0 is lower than NLU threshold 0.4. Predicting fallback action: action_default_fallback
2019-01-07 12:27:45 DEBUG    rasa_core.policies.form_policy  - There is no active form
2019-01-07 12:27:45 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_2_FallbackPolicy
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Predicted next action 'action_default_fallback' with prob 1.00.
2019-01-07 12:27:45 DEBUG    rasa_core.actions.action  - Calling action endpoint to run action 'action_default_fallback'.
DEBUG:rasa_core_sdk.executor:Received request to run 'action_default_fallback'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_default_fallback'
127.0.0.1 - - [2019-01-07 12:27:45] "POST /webhook HTTP/1.1" 200 232 0.001358
2019-01-07 12:27:45 INFO     rasa_core.channels.facebook  - Sending message: Desculpe, não consegui entender, você pode repetir?
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Action 'action_default_fallback' ended with events '['UserUtteranceReverted()']'
2019-01-07 12:27:45 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: Desculpe, não consegui entender, você pode repetir?, data: {
  "elements": null,
  "buttons": null,
  "attachment": null
})'

Thanks, Leonardo

Hi Leonardo,

You could probably improve your intent featurizer by using regex features! You should have a look at rasa-nlu

Hey,

Yes @asokolow good advice. How is your pipeline looking @custodio? You could also use tokenizer_whitespace which strips punctuation and compare performance