Now Available: Integrated Version Control for Rasa X

Today we’re announcing experimental access for a new feature in Rasa X: Integrated Version Control. This feature allows you to securely connect any Git-based code hosting server with Rasa X, to manage version history for training data. Currently, Integrated Version Control is available in Rasa X version 0.23.0, under an experimental feature flag.

What does it do?

Integrated Version Control creates a two-way sync with a remote Git repository that detects diffs between the training data in Rasa X and the remote repo. As you annotate conversations, add stories, and label training examples in Rasa X, you can commit and push those changes to the remote repo. You can also pull down changes from the remote to Rasa X, if changes exist on the Git server but not in Rasa X. Integrated Version Control allows developers to preserve a complete record of each incremental change made to training data.

What are the benefits?

The ability to connect with a remote Git server from within Rasa X significantly streamlines the process of using Rasa X with code hosting platforms and Continuous Integration/Continuous Delivery tooling. Developers can take advantage of a full range of workflows and automations built into and around Git, including end-to-end testing, code reviews, and branch-based development.

Our viewpoint is that industry standard best practices for building software help teams build better AI assistants. We have found the most successful product teams working in conversational AI iterate quickly and apply what they learn from user interactions back into their development process. With the workflows enabled by Integrated Version Control, developers can accelerate the process of continually improving their assistants using Rasa X.

Get Started

Integrated Version Control is compatible with any Git-based code hosting platform, including GitHub, Bitbucket, and GitLab. Get started by downloading Rasa X 0.23 and enabling the Integrated Version Control experiment in your Experiments tab. Check out the documentation for step-by-step instructions.

Share your feedback

Feedback from the community is incredibly important. Tell us what works and what you think could be improved, which tools you use for version control and CI/CD, and the use cases you want supported. We can’t wait for you to try it out.

Happy building!

15 Likes

I’m having problems creating the repo. (Integrated Version Control) I get 401 unauthorized when I try to POST

{“reasons”:[“Authorization header is invalid.”],“exception”:“Unauthorized”}

I have tried decoding the token and it is valid. I get my JSON Web Token from /auth.

My request looks like this

curl --request POST \
     --url https://<URL goes here>/api/projects/default/git_repositories \
     --header 'Authorization: <JWT goes here>' \
     --header 'content-type: application/json' \
     --data-binary @repository.json

I also tried with the API toke, then I get 500 Internal Server Error.

curl --request POST \
     --url https://<URL>/api/projects/default/git_repositories?api_token=<api token goes here> \
     --header 'content-type: application/json' \
     --data-binary @repository.json

Hi @Krogsager,

thanks for trying this out!

I get 401 unauthorized when I try to POST

Sorry, I put the wrong command in the docs. You need the Bearer prefix in the curl:

     --header 'Authorization: Bearer <JWT goes here>' \

I also tried with the API toke, then I get 500 Internal Server Error.

Do you have a server log for me? That’d be great!

JWT version first. I added ‘Bearer’ and get this response:

500 internal server error

The nginx log does not give much

127.0.0.1 - - [29/Nov/2019:15:56:23 +0100] “POST /api/projects/default/git_repositories HTTP/1.1” 500 169 “-” “curl/7.58.0”

I’m running rasa x in a docker container, and nginx on the host machine. I don’t think that nginx can give me much info on an error 500, right? It’s just the messenger.

Can you please do docker-compose logs rasa-x to get the logs from the rasa-x container?

I’m not using the official docker image so my names are a bit different. I have no /app dir

Also, the uri in the log is not https. If that makes a difference.


rasa_vejle              | 2019-11-29 14:56:46 ERROR    sanic.root  - Exception occurred while handling uri: 'http://vejlechat.itkdigital.etek.dk/api/projects/default/git_repositories'                                              
rasa_vejle              | Traceback (most recent call last):                                                                                                                                                                         
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/sanic/app.py", line 942, in handle_request                                                                                                                  
rasa_vejle              |     response = await response                                                                                                                                                                              
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 177, in decorated_function                                                                                         
rasa_vejle              |     return await await_and_return_response(args, kwargs, request)                                                                                                                                          
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 107, in await_and_return_response                                                                                  
rasa_vejle              |     response = await response                                                                                                                                                                              
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 197, in decorated_function                                                                                         
rasa_vejle              |     return await f(request, *args, **kwargs)                                                                                                                                                               
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 48, in add_repository                                                                                          
rasa_vejle              |     git_service = _git(request, project_id)                                                                                                                                                                
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 22, in _git                                                                                                    
rasa_vejle              |     repository_id=repository_id,                                                                                                                                                                           
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/services/git_service.py", line 52, in __init__                                                                                              
rasa_vejle              |     io_utils.create_directory(directory_for_git_clones)                                                                                                                                                    
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasa/utils/io.py", line 380, in create_directory                                                                                                            
rasa_vejle              |     os.makedirs(directory_path)                                                                                                                                                                            
rasa_vejle              |   File "/usr/local/lib/python3.7/os.py", line 211, in makedirs                                                                                                                                             
rasa_vejle              |     makedirs(head, exist_ok=exist_ok)                                                                                                                                                                      
rasa_vejle              |   File "/usr/local/lib/python3.7/os.py", line 221, in makedirs                                                                                                                                             
rasa_vejle              |     mkdir(name, mode)                                                                                                                                                                                      
rasa_vejle              | PermissionError: [Errno 13] Permission denied: '/app'  

I’m not using the official docker image so my names are a bit different. I have no /app dir

Well, that’s gonna make everything a bit complicated :thinking: What prevents you from switching to the offical one? An easy hack could be to create an /app folder as part of your docker image. You should also set an environment variable LOCAL_MODE=false so that Rasa X knows it’s not running locally

What prevents you from switching to the offical one?

I only have one server available for my containerized services. Your official docker image requires a dedicated machine. The install modifies the host machine and messes with my setup.

I lold; LOCAL_MODE=True equals not running locally :stuck_out_tongue:

I’ve added the app dir and added env var. compose logs now return:


rasa_vejle              | 2019-12-02 08:15:14 ERROR    sanic.root  - Exception occurred while handling uri: 'http://vejlechat.itkdigital.etek.dk/api/projects/default/git_repositories'                                              
rasa_vejle              | Traceback (most recent call last):                                                                                                                                                                         
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/sanic/app.py", line 942, in handle_request                                                                                                                  
rasa_vejle              |     response = await response                                                                                                                                                                              
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 177, in decorated_function                                                                                         
rasa_vejle              |     return await await_and_return_response(args, kwargs, request)                                                                                                                                          
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 107, in await_and_return_response                                                                                  
rasa_vejle              |     response = await response                                                                                                                                                                              
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 197, in decorated_function                                                                                         
rasa_vejle              |     return await f(request, *args, **kwargs)                                                                                                                                                               
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 48, in add_repository                                                                                          
rasa_vejle              |     git_service = _git(request, project_id)                                                                                                                                                                
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 22, in _git                                                                                                    
rasa_vejle              |     repository_id=repository_id,                                                                                                                                                                           
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasax/community/services/git_service.py", line 52, in __init__                                                                                              
rasa_vejle              |     io_utils.create_directory(directory_for_git_clones)                                                                                                                                                    
rasa_vejle              |   File "/usr/local/lib/python3.7/site-packages/rasa/utils/io.py", line 380, in create_directory                                                                                                            
rasa_vejle              |     os.makedirs(directory_path)                                                                                                                                                                            
rasa_vejle              |   File "/usr/local/lib/python3.7/os.py", line 221, in makedirs                                                                                                                                             
rasa_vejle              |     mkdir(name, mode)                                                                                                                                                                                      
rasa_vejle              | PermissionError: [Errno 13] Permission denied: '/app/git' 

User/group ownership of the dir matches the rest of the project-

Your official docker image requires a dedicated machine.

That’s not correct. With docker-compose you can run on a single machine.

I lold; LOCAL_MODE=True equals not running locally :stuck_out_tongue:

sorry, that was a type :laughing: It’s LOCAL_MODE=false :smiley:

I’ve added the app dir and added env var. compose logs now return:

Can you make sure user 1001 or group 0 has read / write permissions to this directory?

I nearly lost my mind today. Setting the env var LOCAL_MODE=false broke rasa.

Rasa x crashes on start - missing psycopg2

Regarding the official docker image, the instructions here on Rasa X downloads and runs this ansible playbook, which I’m wary of.

@Tobias_Wochinger Why might LOCAL_MODE=false break psycopg2?

Hi @Krogsager,

there is also documentation how to use the official docker image without using the ansible script :slight_smile: Please see the docs in this section: Deploy to a Server .

I nearly lost my mind today. Setting the env var LOCAL_MODE=false broke rasa.

Rasa x crashes on start - missing psycopg2

Oh dear :grimacing: Sorry for the hard time!

Rasa X in server mode uses postgresql as a database since sqlite is not suited for a production setup. In your docker image you have to install libpq-dev and you should install rasa with pip install rasa-x[sql] --extra-index-url https://pypi.rasa.com/simple (or simply do pip install psycopg2~=2.8).

I previously tried to pip install psycopg2 but that threw another error, so I’m doing the “right” thing instead:

I did RUN pip install rasa-x[sql]==0.23.2 --extra-index-url https://pypi.rasa.com/simple in a new container but I get a new error:

File "/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py", line 126, in connect                                                                                                                                               
  conn = _connect(dsn, connection_factory=connection_factory, **kwasync)                                                                                                                                                           
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "db" to address: Temporary failure in name resolution

Should I cherry-pick something from the manual configuration?