I want to use HuggingFace language models in my pipeline how can I add to my pipeline and how to use them. I want my bot in Multilingual (English-Arabic) switch automatically instead of manually adding (Intents and Responses).I have heard that (HFT) have a langugage model which support Arabic. I don’t know how to use that.
SpaCy and MITIE didn’t support Arabic. Here is my project what I did to make changes.
version: "2.0" nlu: - intent: greet examples: | - hey - hello - hi - hello there - good morning - good evening - moin - hey there - let's go - hey dude - goodmorning - goodevening - good afternoon
# Configuration for Rasa NLU. # https://rasa.com/docs/rasa/nlu/components/ language: en pipeline: # # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model. # # If you'd like to customize it, uncomment and adjust the pipeline. # # See https://rasa.com/docs/rasa/tuning-your-model for more information. - name: HFTransformersNLP model_name: "bert" model_weights: "rasa/LaBSE" cache_dir: null - name: "LanguageModelTokenizer" "intent_tokenization_flag": False "intent_split_symbol": "_" - name: DIETClassifier epochs: 100 constrain_similarities: true - name: EntitySynonymMapper - name: ResponseSelector epochs: 100 constrain_similarities: true - name: FallbackClassifier threshold: 0.3 ambiguity_threshold: 0.1 # Configuration for Rasa Core. # https://rasa.com/docs/rasa/core/policies/ policies: # # No configuration for policies was provided. The following default policies were used to train your model. # # If you'd like to customize them, uncomment and adjust the policies. # # See https://rasa.com/docs/rasa/policies for more information. # - name: MemoizationPolicy # - name: RulePolicy # - name: UnexpecTEDIntentPolicy # max_history: 5 # epochs: 100 # - name: TEDPolicy # max_history: 5 # epochs: 100 # constrain_similarities: true
After running (rasa shell)
Bot loaded. Type a message and press enter (use '/stop' to exit): Your input -> hi D:\Anaconda3\envs\bot\lib\site-packages\rasa\shared\utils\io.py:97: UserWarning: Interpreter parsed an intent 'hi' which is not defined in the domain. Please make sure all intents are listed in the domain. More info at https://rasa.com/docs/rasa/domain 2021-08-20 11:37:51 ERROR asyncio - Task exception was never retrieved future: <Task finished name='Task-2' coro=<configure_app.<locals>.run_cmdline_io() done, defined at D:\Anaconda3\envs\bot\lib\site-packages\rasa\core\run.py:131> exception=TimeoutError()> Traceback (most recent call last): File "D:\Anaconda3\envs\bot\lib\site-packages\rasa\core\run.py", line 135, in run_cmdline_io await console.record_messages( File "D:\Anaconda3\envs\bot\lib\site-packages\rasa\core\channels\console.py", line 182, in record_messages async for response in bot_responses: File "D:\Anaconda3\envs\bot\lib\site-packages\rasa\core\channels\console.py", line 137, in _send_message_receive_stream async for line in resp.content: File "D:\Anaconda3\envs\bot\lib\site-packages\aiohttp\streams.py", line 39, in __anext__ rv = await self.read_func() File "D:\Anaconda3\envs\bot\lib\site-packages\aiohttp\streams.py", line 338, in readline await self._wait("readline") File "D:\Anaconda3\envs\bot\lib\site-packages\aiohttp\streams.py", line 306, in _wait await waiter File "D:\Anaconda3\envs\bot\lib\site-packages\aiohttp\helpers.py", line 656, in __exit__ raise asyncio.TimeoutError from None asyncio.exceptions.TimeoutError