Preparing Test Set

Rasa Open Source
1

Hi guys,

I have a quick question. When we prepare a test set, should we annotate the intents and entities in it, as same as the training set, or we should not annotate the intents and entities in it? For example in the test set:

intent: addressEdit

  • please, let me add my new address. /entities are explicitly shown here

or

  • please, let me add my new address.

Which one is correct?

My second question is: Let’s say we have several utterances almost the same expression but the ending of the sentences are changing. Should we keep them as in the below example or only “can you help me to update insurance address” is enough in the training data.

  • can you help me to update insurance address on my account
  • can you help me to update insurance address on my page
  • can you help me to update insurance address on my insurance
  • can you help me to update insurance address on vehicle insurance
  • can you help me to update insurance address on my car insurance
  • can you help me to update insurance address on my auto insurance

My last and crucial question is:

for example in training set, there is an intent whose name is “addressEdit” this intent includes billing address, mailing address but does not include email address. Let’s say email address is in emailEdit intent, which is another intent. When I design the TEST set, the below example is correct or not.

addressEdit

TEST SET: ##addressEdit

  • I changed my mailing address
  • I have a residence
  • I want to update my email address

So should I include an email address in addressEdit TEST set to check if the model can classify it correctly or it should be under emailEdit intent in the Test Set. Unfortunately, I could not find how to design a test set so far including in tutorials

Thank you very much

Sincerely, Serdar