DIET unable to generalize entity extraction

Rasa Open Source
1

I have created an intent with entity annotations for amount_of_money using enitity “roles” to be either “to” or “from”. here are a couple of examples

am looking for [mobiles](product_type) in range between [1k]{"entity": "amount-of-money", "role": "from"} to [2k]{"entity": "amount-of-money", "role": "to"} dollars

mobiles between [2k]{"entity": "amount-of-money", "role": "from"} to [3k]{"entity": "amount-of-money", "role": "to"}

…and many more such examples.

But when I test it with a new amount say “5k” that is tagged in examples only at one place as “to”, it always get role assigned to it as “to”. Whereas as per my testin utterance it should come in role “from”.

1 Like
2

Hi @mriganktiwari

Thanks for the question. We have definitely received this as feedback from some of our users before as well. We recently tweaked the role prediction algorithm and implemented the tweak on an experimental branch. Would you be willing to try it out?

You’ll have to install Rasa from source on that branch. Here are the installation docs to help you with that. (Don’t forget to switch to the roles-attn branch after cloning the repo :slight_smile: )

Do let us know about your findings / observations on how the performance of the model trained on this branch is compared to the performance of your earlier model.

1 Like
3

Hi @dakshvar22,

Sure, let me give it a try and get back to you with findings.

Thanks for letting me know about the experiments you guys are doing.

Mrigank

4

Hi @dakshvar22 , I am getting below error when installed rasa from “roles-attn” branch and trying to train.

/home/test/playground/rasa/rasa/utils/tensorflow/models.py:298 train_on_batch  *
    prediction_loss = self.batch_loss(batch_in)
/home/test/playground/rasa/rasa/nlu/classifiers/diet_classifier.py:1443 batch_loss  *
    sequence_lengths = self._get_sequence_lengths(
/home/test/playground/rasa/rasa/utils/tensorflow/models.py:1154 _get_sequence_lengths  *
    sequence_lengths = tf.ones([batch_dim], dtype=tf.int32)
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
    return target(*args, **kwargs)
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py:3041 ones
    output = _constant_if_small(one, shape, dtype, name)
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py:2732 _constant_if_small
    if np.prod(shape) < 1000:
<__array_function__ internals>:5 prod
    
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/numpy/core/fromnumeric.py:3051 prod
    return _wrapreduction(a, np.multiply, 'prod', axis, dtype, out,
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/numpy/core/fromnumeric.py:86 _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
/home/test/miniconda3/envs/erasa/lib/python3.8/site-packages/tensorflow/python/framework/ops.py:845 __array__
    raise NotImplementedError(
    
NotImplementedError: Cannot convert a symbolic Tensor (strided_slice_6:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported
  • Numpy version I have is 1.21.2

Kindly help.

5

I am unable to reproduce the error. Did you start from a fresh python environment and installed rasa from source?

6

Yes, I was trying with a fresh python env and from source.

Another question if you could help me with, in which version are these “role prediction” changes supposed to be released officially?

7

Could you please confirm in which version are these “role prediction” changes will be released?

1 Like