Here!
As we all know, rasa handle one model by on process.I created a project named “rasa-z” which I want to serve a large number of account.
Features:
1>Support GPU and CPU separation.
GPU assets is not so import for model parsing but important for training.
2>Support cluster master
Based on npm ‘pm2’.You can also use K8s.
It’s optional
3>Support easy API
Just train parse and bot CURD.
When bot is created,server will return APP_ID
You can parse QA by passing APP_ID and Q.
If you want to train data,just pass [{ Q:[{ q:’’ },{q:"}] },A:’ ']
The server will train it and use ResponseSelector for chitchat/faq automatically
You won’t pass config\domain\intent or any other params.
4>GUI that similar to RASA multi bot version .Not be related with RASA/RASA X.
5>Support custom bot or template bot.
6>Support easy faq bot or complex story bot.
7>Support private or public cloud server.
For supprting the RASA team. I will not show all my code.
But it’s useful if u are not deal with multi bot and cloud bot server.
I also hope RASA team can deploy their multi bot version.If needed, I can support RASA team to ceate this issue.
The following is the relevant diagram | implementation description | some code implementation.
High level Architecture:
The basic functions of the master and node include: load balancing, rest interface, management GUI, native management, and creating rasa NLU services.
The role of the master is mainly to receive and allocate external unified requests. In the case of a single machine, the master server can receive and process requests, or in the case of a cluster, a node can be assigned to process external requests. Nodes are generally not exposed.
② DB: it is mainly used to store the uniformly received request log and record the road address of each node
③ NLU training server: in essence, it is also a rasa service, which is used solely to use the GPU training model. Since the training itself is stateless, the load balancing of multiple GPU servers can be realized through nginx
④ Rasa node: each rasa node is a rasa service instance, with independent internal IP and ports, and provides rasa API services. Rasa node is created through rasa-c node and mapped externally.
⑤ / ⑥ third party servers and clients. You can use appid to access and use different rasa services through the unified interface exposed by rasa master.
⑦ Same domain and same IP: used on a virtualized stand-alone computer. At this point, rasa-c node is equivalent to rasa-c master.
⑧ Different IP addresses in the same domain: generally used for the expansion of servers in a cluster / environment. For example, the horizontal expansion under the public cloud.
⑨ Different domain same / different IP: for example, a node deployed in the customer’s private cloud can be a public IP. Can be accessed externally.
⑩ Nodes can be set to stateless. Therefore, if necessary, different masters can allocate and point to the same rasa-c node. That is, a rasa-c node is used by multiple masters. This can be used for
Master migration. For example, after copying the master node, deactivate the original master node.
For the reuse of resources, for example, the privatized deployed master can share the nodes of the public cloud
Hybrid cloud. For example, the public cloud master can access the nodes of the private cloud. Implement hybrid cloud.
GUI eg. based on rasa-ui(EN)&rasa-ui-zh.But improved train func and train data and added more custom options
1>multi bot curd
Auto create rasa server and master it in cluster.
1>Creating bot and rasa server.auto use free port.
3>Easy query
The file domain:
1>level 1:front ui and server
2>level 2:backend and multi app
apps:multi rasa menu.server will just use default model file to parse.
template:the default bot template.You can custom it .
3>level 3:rasa-instance .
It’s created by server.but u can instead it by your custom bot,which can by used for customized requirements of users.
Code implementation
1>GUI: please see rasa-ui or rasa-ui-zh on github.I used their framework.The frontend is based on AngularJS ,but it’s friendly to inexperienced people.You won’t change to much.
2>PM2 cluster master
The backend is based on nodejs.You can use npm ‘shelljs’ to create scrpits and run it by ‘pm2 start xxx.sh/js/**’
first , copy rasa folder under server/apps from your template folder server/template
second, creat shell script which implement env config.just like rasa server port .
By default,it will just use model but not use domain or other file.The data is stored on rasa-z sqlite db(you can change it to mysql).The model training server(can be one machine or different machine) will handle server data training post and store the model file under your apps/rasa-name/models.
Now I’m working on improving robustness.
I hope this issue can help some people. 