Hello all, I was trying to run an evaluation using cross-validation, but I’m getting the following errors during each fold. Has someone run into this before and how does one resolve it?
/opt/venv/lib/python3.7/site-packages/sklearn/metrics/_classification.py:1221: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use zero_division parameter to control this behavior.
For additional information, I’ve got an NLU-only model and I’m only using it for intent classification (no entities yet). I’m using DIET in my config (see below) -
# Current pipeline configuration for the NLU language: "en" pipeline: - name: ConveRTTokenizer intent_tokenization_flag: true # Set to true to spot multiple intents case_sensitive: False intent_split_symbol: "+" - name: ConveRTFeaturizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier epochs: 100 - name: EntitySynonymMapper - name: ResponseSelector epochs: 100 # Configuration for Rasa Core. # https://rasa.com/docs/rasa/core/policies/ policies: - name: MemoizationPolicy - name: TEDPolicy - name: MappingPolicy
Finally, this error appears before generating the confusion matrix. Would also appreciate some help here -
/opt/venv/lib/python3.7/site-packages/numpy/core/_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
Many thanks for your help!