Failing at intent classification

I am training my model which has following data but I am failing at intent classification. can anyone suggest why its happening and suggest better parameter for intent classifier to improve dection.

intent Examples of intents
budget_details 13
goodbye 24
affirm 29
deny 36
greet 42
product_submission 2776
request_waiver 2976
customer_details 3011
product_details 4913
product_information 26516

@akelad @JiteshGaikwad @Juste @JulianGerhard

Hi @shubham

this is a bit vague to be honest. Could you please provide some verbose information?

  1. Are you able to train the bot?
  2. Are you able to start the bot?
  3. Could you start the bot with -vv flag and post the output here?

Besides it seems a bit unbalanced where “a bit” means a lot. Is your classifier working but you are not satisfied with its performance? If so, please post your pipeline, such that we can analyze the problem properly!

Regards Julian

Hey @shubham, may I see the confusion matrix produced by rasa test nlu? Essentially, I want to see how exactly your intent classifier fails.

language: en

pipeline:

  • name: “SpacyNLP” model: “en_core_web_lg”

  • name: “SpacyTokenizer”

  • name: “SpacyFeaturizer”

  • name: “RegexFeaturizer”

  • name: DucklingHTTPExtractor url: http://localhost:8000 dimensions:

    • time
    • amount-of-money
  • name: “CRFEntityExtractor” features: [[“low”, “title”,“pos”, “pos2”], [“bias”, “low”, “prefix5”, “prefix2”, “suffix5”,“digit”, “suffix3”,“suffix2”,“upper”, “title” ,“pattern”], [“low”, “title”, “upper”, “pos”, “pos2”]] BILOU_flag: true max_iterations: 50 L1_c: 0.1 L2_c: 0.1

  • name: “EntitySynonymMapper”

  • name: CountVectorsFeaturizer

    OOV_token: OOV

    token_pattern: (?u)\b\w+\b

  • name: “EmbeddingIntentClassifier”

    “hidden_layers_sizes_a”: [256, 128]

    “hidden_layers_sizes_b”: []

    “batch_size”: 10 “epochs”: 300

    “embed_dim”: 20

    “mu_pos”: 0.8
    “mu_neg”: -0.4
    “similarity_type”: “cosine”
    “num_neg”: 20

    “use_max_sim_neg”: true
    “random_seed”: 50

    “C2”: 0.002

    “C_emb”: 0.9

    “droprate”: 0.2

    “intent_tokenization_flag”: true

    “intent_split_symbol”: “+”

    “evaluate_every_num_epochs”: 10
    “evaluate_on_num_examples”: 1000

policies:

  • name: MemoizationPolicy
  • name: KerasPolicy
  • name: MappingPolicy
  • name: FormPolicy

Hey @shubham, thanks for posting the pipeline, I think it will come handy a bit later.

For now, can, please, you explain what is not working and ideally provide the kind of information (either error messages or the intent confusion matrix, etc.) that would help me or @JulianGerhard help you, as we explained earlier? Thanks.

1 Like