Today we’re announcing experimental access for a new feature in Rasa X: Integrated Version Control. This feature allows you to securely connect any Git-based code hosting server with Rasa X, to manage version history for training data. Currently, Integrated Version Control is available in Rasa X version 0.23.0, under an experimental feature flag.
What does it do?
Integrated Version Control creates a two-way sync with a remote Git repository that detects diffs between the training data in Rasa X and the remote repo. As you annotate conversations, add stories, and label training examples in Rasa X, you can commit and push those changes to the remote repo. You can also pull down changes from the remote to Rasa X, if changes exist on the Git server but not in Rasa X. Integrated Version Control allows developers to preserve a complete record of each incremental change made to training data.
What are the benefits?
The ability to connect with a remote Git server from within Rasa X significantly streamlines the process of using Rasa X with code hosting platforms and Continuous Integration/Continuous Delivery tooling. Developers can take advantage of a full range of workflows and automations built into and around Git, including end-to-end testing, code reviews, and branch-based development.
Our viewpoint is that industry standard best practices for building software help teams build better AI assistants. We have found the most successful product teams working in conversational AI iterate quickly and apply what they learn from user interactions back into their development process. With the workflows enabled by Integrated Version Control, developers can accelerate the process of continually improving their assistants using Rasa X.
Get Started
Integrated Version Control is compatible with any Git-based code hosting platform, including GitHub, Bitbucket, and GitLab. Get started by downloading Rasa X 0.23 and enabling the Integrated Version Control experiment in your Experiments tab. Check out the documentation for step-by-step instructions.
Share your feedback
Feedback from the community is incredibly important. Tell us what works and what you think could be improved, which tools you use for version control and CI/CD, and the use cases you want supported. We can’t wait for you to try it out.
I’m running rasa x in a docker container, and nginx on the host machine. I don’t think that nginx can give me much info on an error 500, right? It’s just the messenger.
I’m not using the official docker image so my names are a bit different. I have no /app dir
Well, that’s gonna make everything a bit complicated What prevents you from switching to the offical one? An easy hack could be to create an /app folder as part of your docker image. You should also set an environment variable LOCAL_MODE=false so that Rasa X knows it’s not running locally
What prevents you from switching to the offical one?
I only have one server available for my containerized services. Your official docker image requires a dedicated machine. The install modifies the host machine and messes with my setup.
I lold; LOCAL_MODE=True equals not running locally
I’ve added the app dir and added env var.
compose logs now return:
rasa_vejle | 2019-12-02 08:15:14 ERROR sanic.root - Exception occurred while handling uri: 'http://vejlechat.itkdigital.etek.dk/api/projects/default/git_repositories'
rasa_vejle | Traceback (most recent call last):
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/sanic/app.py", line 942, in handle_request
rasa_vejle | response = await response
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 177, in decorated_function
rasa_vejle | return await await_and_return_response(args, kwargs, request)
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 107, in await_and_return_response
rasa_vejle | response = await response
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/api/decorators.py", line 197, in decorated_function
rasa_vejle | return await f(request, *args, **kwargs)
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 48, in add_repository
rasa_vejle | git_service = _git(request, project_id)
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/api/blueprints/git.py", line 22, in _git
rasa_vejle | repository_id=repository_id,
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasax/community/services/git_service.py", line 52, in __init__
rasa_vejle | io_utils.create_directory(directory_for_git_clones)
rasa_vejle | File "/usr/local/lib/python3.7/site-packages/rasa/utils/io.py", line 380, in create_directory
rasa_vejle | os.makedirs(directory_path)
rasa_vejle | File "/usr/local/lib/python3.7/os.py", line 221, in makedirs
rasa_vejle | mkdir(name, mode)
rasa_vejle | PermissionError: [Errno 13] Permission denied: '/app/git'
User/group ownership of the dir matches the rest of the project-
Rasa X in server mode uses postgresql as a database since sqlite is not suited for a production setup. In your docker image you have to install libpq-dev and you should install rasa with pip install rasa-x[sql] --extra-index-url https://pypi.rasa.com/simple (or simply do pip install psycopg2~=2.8).
I previously tried to pip install psycopg2 but that threw another error, so I’m doing the “right” thing instead:
I did RUN pip install rasa-x[sql]==0.23.2 --extra-index-url https://pypi.rasa.com/simple in a new container but I get a new error:
File "/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "db" to address: Temporary failure in name resolution
Should I cherry-pick something from the manual configuration?
I was looking forward to trying out the integrated version control. But I fail at the very beginning. I can’t find the way to "select Experimental → Integrated Version Control ".
I installed rasa-x 0.23.4 using pip and run it in the local model on my machine. However, I don’t see the menu option “Experimental”.
What am I doing wrong?
Integrated Version Control is only available in the server edition of Rasa X which is deployed using the docker images. In local mode, you can just git add, commit, and push your changes yourself.
Just passing by to share a (perhaps) useful advice. When adding your repository to Rasa X, on Step 2, there is an important step that’s missing in the docs. That is, when you paste your private key, you need to replace new lines to '\n' character as that’s needed to successfully parse the key from JSON.
Before that I was getting this error (makes it easy for people to find this out afterwards):
{
"version": "0.23.5.dev391+g95e6e50",
"status": "failure",
"message": "Insufficient permissions for remote repository.",
"reason": "RepositoryCreationFailed",
"details": {
"args": [
"Given repository credentials don't provide write permissions to the repository. Please make sure the ssh key is correct and the administrator of the remote repository gave you the required permissions."
]
},
"help": null,
"code": 422
}
conversion can be accomplished by a few lines on an IPython session on the same directory as the private key:
replaces_nl = lambda line: line.replace('\n', '\\n')
concat_replace = lambda a, b: replaces_nl(f'{a}{b}')
with open('git-deploy-key') as f:
one_liner = reduce(concat_replace, f.readlines())
with open('git-deploy-key-oneliner', 'w') as f:
f.write(one_liner)
seizing any opportunity to use functools
You can now paste git-deploy-key-oneliner as a one-liner string on repository.json and follow through to sending the request to link both services (Step 3).
And after going through all steps, I had to recreate rasa-x service in order to integrations apply de-facto:
I’ve been trying to activate Integrated Version Control and now I believe I overcome the problems others had so far in this thread (ie. successfully activated server mode, checked the experimental box, successfully connected to a sql tracker store, successfully uploaded the git ssh key).
But the next step according to the docs would be to see the version control status on the side bar, showing sync status with the git repository. But unfortunately the Rasa-X interface did not change at all. I even tried to modify something in the training data to see if it would do anything, to no avail.
Logs are quite clean. No error message related to version control. No message when the key is uploaded. Should there be one? I start rasa with -v. When the key is uploaded the response to the client is:
The following is the only leftover warning in my setup, could this be related?
/build/lib/python3.6/site-packages/rasax/community/services/event_service.py:1006: UserWarning: Could not find ‘event_broker’ section in endpoint config file at path ‘endpoints.yml’.
f"Could not find ‘{event_broker_key}’ section in "
Any clue how to troubleshoot the sidebar not appearing at all after all steps being completed?
I had the same error and tried to solve it the way you wrote, but I’m still getting it.
This is how my repository.json file looks like:
And this is the error that I get:
{"version":"0.24.1","status":"failure","message":"Insufficient permissions for remote repository.","reason":"RepositoryCreationFailed","details":{"args":["Given repository credentials don't provide write permissions to the repository. Please make sure the ssh key is correct and the administrator of the remote repository gave you the required permissions."]},"help":null,"code":422}
Hey @tiziano, please make sure all these items are covered:
you created a brand new SSH key-pair for this purpose
this public key is registered on smartpdt-chatbot repository under “Settings → Deploy Keys”
that you created git-deploy-oneliner from this private key and it is still a valid key (meaning you didn’t edit anything accidentaly)
As a matter of fact, it seems to me that you are using a PEM private key format. I would definitely (go back to step 1 and create one) try using a OpenSSH key format. I’m 100% not sure about this specification but I would give it a try since worked for me.
What prevents you from switching to the offical one? An easy hack could be to create an 
It’s 
Please see the docs in this section: 
Sorry for the hard time!
