Need help please!

i’ve solved thank you i have another question , i have alwayes this result however i changed the message

always detecting a postive sentiment with confidence 0.333

{‘intent’: {‘name’: ‘insultes’, ‘confidence’: 0.35888357915887636}, ‘entities’: [{‘value’: ‘pos’, ‘confidence’: 0.3333333333333333, ‘entity’: ‘sentiment’, ‘extractor’: ‘sentiment_extractor’}], ‘intent_ranking’: [{‘name’: ‘insultes’, ‘confidence’: 0.35888357915887636}, {‘name’: ‘salutations’, ‘confidence’: 0.07922486366609584}, {‘name’: ‘au_revoir’, ‘confidence’: 0.06963384023417138}, {‘name’: ‘out’, ‘confidence’: 0.057168641232126574}, {‘name’: ‘positive_answer’, ‘confidence’: 0.05699548462320999}, {‘name’: ‘congrats’, ‘confidence’: 0.05146011802312083}, {‘name’: ‘salaire’, ‘confidence’: 0.05105493986389435}, {‘name’: ‘annulation’, ‘confidence’: 0.04765725667849151}, {‘name’: ‘negative_answer’, ‘confidence’: 0.04180718673910084}, {‘name’: ‘merci’, ‘confidence’: 0.03432302984684233}], ‘text’: ‘connard’}

Hi @m2cci-sba. So to be clear, your issue is that no matter what you send, your custom sentiment analyser always returns positive with confidence 0.333?

yes !!!

Sounds like something is wrong within the custom sentiment analyser component then, does the code work on its own?

yes it’s working , i have just one stupid question :smiley: how to do the connexion between my labels.txt (that contain the sentiments ) and my data nlu because i’ve remplaced

training_data = training_data.training_examples #list of Message objects tokens = [list(map(lambda x: x.text, t.get(‘tokens’))) for t in training_data]

with :slight_smile:

nlu = nlu tokens = [list(map(lambda x: x.text, t.get(‘tokens’))) for t in nlu]

nlu is my the data nlu folder .

I’ve created also a registry file :slight_smile: from rasa_nlu.customcomponents.sentiment import MySentimentAnalyzer

from data import sentiment

all import needed

component_classes = [

all rasa components added no change

MySentimentAnalyzer

]

here is my chatbot-nlp folder

my data contains nlu ans stories .

my config file :slight_smile:# Configuration for Rasa NLU.

Components

language: fr_core_news_md pipeline:

  • name: “SpacyNLP”
  • name: “SpacyTokenizer”
  • name: “sentiment.SentimentAnalyzer”
  • name: “SpacyFeaturizer”
  • name: “WhitespaceTokenizer”
  • name: “RegexFeaturizer”
  • name: “CRFEntityExtractor”
  • name: “EntitySynonymMapper”
  • name: “CountVectorsFeaturizer”
  • name: “SklearnIntentClassifier”

Configuration for Rasa Core.

Policies

policies:

  • name: MemoizationPolicy
  • name: KerasPolicy
  • name: MappingPolicy
  • name: FormPolicy
  • name: “FallbackPolicy” nlu_threshold: 0.25 core_threshold: 0.3 fallback_action_name: “action_my_fallback”

labels:

pos pos neu neu neg neg

sentiment.py

from rasa.nlu.components import Component from rasa.nlu import utils from rasa.nlu.model import Metadata

import nltk from nltk.classify import NaiveBayesClassifier import os

import typing from typing import Any, Optional, Text, Dict

SENTIMENT_MODEL_FILE_NAME = “sentiment_classifier.pkl”

class SentimentAnalyzer(Component): “”“A custom sentiment analysis”"" name = “sentiment” provides = [“entities”] requires = [“tokens”] defaults = {} language_list = [“fr_core_news_md”] print(‘initialised the class’)

def __init__(self, component_config=None):
    super(SentimentAnalyzer, self).__init__(component_config)
def train(self, training_data, cfg, **kwargs):

    """Load the sentiment polarity labels from the text
       file, retrieve training tokens and after formatting
       data train the classifier."""
    with open('labels.txt','r') as f:
        labels = f.read().splitlines()
    training_data = training_data.training_examples #list of Message objects
    tokens = [list(map(lambda x: x.text, t.get('tokens'))) for t in training_data]
    processed_tokens = [self.preprocessing(t) for t in tokens]
    labeled_data = [(t, x) for t,x in zip(processed_tokens, labels)]
    self.clf = NaiveBayesClassifier.train(labeled_data)
def convert_to_rasa(self, value, confidence):
    """Convert model output into the Rasa NLU compatible output format."""

    entity = {"value": value,
              "confidence": confidence,
              "entity": "sentiment",
              "extractor": "sentiment_extractor"}

    return entity
def preprocessing(self, tokens):
    """Create bag-of-words representation of the training examples."""
    
    return ({word: True for word in tokens})
def process(self, message, **kwargs):
    """Retrieve the tokens of the new message, pass it to the classifier
        and append prediction results to the message class."""
    
    if not self.clf:
        # component is either not trained or didn't
        # receive enough training data
        entity = None
    else:
        tokens = [t.text for t in message.get("tokens")]
        tb = self.preprocessing(tokens)
        pred = self.clf.prob_classify(tb)

        sentiment = pred.max()
        confidence = pred.prob(sentiment)

        entity = self.convert_to_rasa(sentiment, confidence)

        message.set("entities", [entity], add_to_output=True)
def persist(self, file_name, model_dir):
    """Persist this model into the passed directory."""
    classifier_file = os.path.join(model_dir, SENTIMENT_MODEL_FILE_NAME)
    utils.json_pickle(classifier_file, self)
    return {"classifier_file": SENTIMENT_MODEL_FILE_NAME}

@classmethod
def load(cls,
         meta: Dict[Text, Any],
         model_dir=None,
         model_metadata=None,
         cached_component=None,
         **kwargs):
    file_name = meta.get("classifier_file")
    classifier_file = os.path.join(model_dir, file_name)
    return utils.json_unpickle(classifier_file)

where and why did you change this? I think that might be causing problems since you’re not using the training data to train the model?

I should use my data nlu to connect it with labels, and I made those changes in the sentiment.py file but I also try to run it without changing the file but still not working how can I connect my data file to sentiment.py and labels. Txt please

Right, but you only changed the code in your custom code right, not in the rest of the code?

Yes I added the custom component the same as the tutorial, registry file, labels.txt and I’ve added tlin the pipeline the new custom component that’s it