I’m about to working on Bots that support for both English and Japanese, along the way I face the problems with Japanese Text. I now use “tensorflow” as pipeline. I run everything as default then Bots seem to not working fine. I read the topics around and I came to know that Japanese needed special case. Default tokenization is not working, I need something like “MeCab”. I try to look around but cannot see a way how to really put them together.
Anyone could guide me a bit in this is really appreciate ??