Since Deepspeech is not a real time Speech2Text engine, it won’t be as intuitive as something like Google Assistant that gives real time text of your speech as you speak. Also putting speech to text model on a mobile app would be tough as the models have to be very concise. I suggest you host your bot servers separately and build an app that calls your bot APIs.
The better way to do it would be to use an existing bot framework like Google Assistant and link your Rasa bot with it. You can make use of Google Actions console.
Here is a great blog by @Juste from the Rasa team on integrating Rasa bot with Google Assistant: Going beyond hey google, building a rasa powered google assistant