Deployment to Heroku(CI with github) - Slug size exceeded - require small rasa docker image

Hi ,

When trying to deploy bot to heroku with github, I get error slugsize exceeded.

  Installing collected packages: ruamel.yaml.clib, ruamel.yaml, tqdm, python-crfsuite, tabulate, six, sklearn-crfsuite, numpy, scipy, joblib, scikit-learn, future, idna, certifi, chardet, urllib3, requests, requests-toolbelt, PyJWT, webexteamssdk, dnspython, pymongo, async-timeout, typing-extensions, attrs, multidict, yarl, idna-ssl, aiohttp, slackclient, gast, cloudpickle, decorator, tensorflow-probability, mattermostwrapper, colorclass, zipp, importlib-metadata, pyrsistent, jsonschema, ujson, tensorflow-estimator, redis, pyparsing, packaging, pydot, pytz, python-dateutil, httptools, aiofiles, websockets, uvloop, h11, immutables, contextvars, sniffio, rfc3986, hstspreload, hyperframe, hpack, h2, httpx, sanic, sanic-plugins-framework, sanic-cors, docutils, jmespath, botocore, s3transfer, boto3, pyasn1, pyasn1-modules, httplib2, rsa, oauth2client, tzlocal, apscheduler, pycparser, cffi, cryptography, tornado, python-telegram-bot, rocketchat-API, absl-py, grpcio, oauthlib, requests-oauthlib, cachetools, google-auth, google-auth-oauthlib, protobuf, werkzeug, markdown, tensorboard, google-pasta, astor, opt-einsum, keras-preprocessing, wrapt, h5py, keras-applications, termcolor, tensorflow, tensorflow-addons, python-engineio, python-socketio, wcwidth, prompt-toolkit, terminaltables, sanic-jwt, humanfriendly, coloredlogs, async-generator, colorhash, pika, questionary, psycopg2-binary, kiwisolver, cycler, matplotlib, rasa-sdk, networkx, PyYAML, docopt, pykwalify, SQLAlchemy, tensorflow-hub, jsonpickle, pysocks, twilio, fbmessenger, regex, greenlet, gevent, kafka-python, rasa
   Successfully installed PyJWT-1.7.1 PyYAML-5.3.1 SQLAlchemy-1.3.19 absl-py-0.9.0 aiofiles-0.5.0 aiohttp-3.6.2 apscheduler-3.6.3 astor-0.8.1 async-generator-1.10 async-timeout-3.0.1 attrs-19.3.0 boto3-1.14.57 botocore-1.17.57 cachetools-4.1.1 certifi-2020.6.20 cffi-1.14.2 chardet-3.0.4 cloudpickle-1.3.0 colorclass-2.2.0 coloredlogs-10.0 colorhash-1.0.2 contextvars-2.4 cryptography-3.1 cycler-0.10.0 decorator-4.4.2 dnspython-1.16.0 docopt-0.6.2 docutils-0.15.2 fbmessenger-6.0.0 future-0.18.2 gast-0.4.0 gevent-1.5.0 google-auth-1.21.1 google-auth-oauthlib-0.4.1 google-pasta-0.2.0 greenlet-0.4.16 grpcio-1.31.0 h11-0.8.1 h2-3.2.0 h5py-2.10.0 hpack-3.0.0 hstspreload-2020.9.9 httplib2-0.18.1 httptools-0.1.1 httpx-0.9.3 humanfriendly-8.2 hyperframe-5.2.0 idna-2.10 idna-ssl-1.1.0 immutables-0.14 importlib-metadata-1.7.0 jmespath-0.10.0 joblib-0.16.0 jsonpickle-1.4.1 jsonschema-3.2.0 kafka-python-1.4.7 keras-applications-1.0.8 keras-preprocessing-1.1.0 kiwisolver-1.2.0 markdown-3.2.2 matplotlib-3.2.2 mattermostwrapper-2.2 multidict-4.7.6 networkx-2.4 numpy-1.19.1 oauth2client-4.1.3 oauthlib-3.1.0 opt-einsum-3.3.0 packaging-20.4 pika-1.1.0 prompt-toolkit-2.0.10 protobuf-3.13.0 psycopg2-binary-2.8.6 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pydot-1.4.1 pykwalify-1.7.0 pymongo-3.8.0 pyparsing-2.4.7 pyrsistent-0.17.1 pysocks-1.7.1 python-crfsuite-0.9.7 python-dateutil-2.8.1 python-engineio-3.12.1 python-socketio-4.5.1 python-telegram-bot-12.8 pytz-2019.3 questionary-1.5.2 rasa-1.10.11 rasa-sdk-1.10.2 redis-3.5.3 regex-2020.6.8 requests-2.24.0 requests-oauthlib-1.3.0 requests-toolbelt-0.9.1 rfc3986-1.4.0 rocketchat-API-1.3.1 rsa-4.6 ruamel.yaml-0.16.12 ruamel.yaml.clib-0.2.2 s3transfer-0.3.3 sanic-19.12.2 sanic-cors-0.10.0.post3 sanic-jwt-1.4.1 sanic-plugins-framework-0.9.3 scikit-learn-0.22.2.post1 scipy-1.5.2 six-1.15.0 sklearn-crfsuite-0.3.6 slackclient-2.8.2 sniffio-1.1.0 tabulate-0.8.7 tensorboard-2.1.1 tensorflow-2.1.1 tensorflow-addons-0.7.1 tensorflow-estimator-2.1.0 tensorflow-hub-0.8.0 tensorflow-probability-0.9.0 termcolor-1.1.0 terminaltables-3.1.0 tornado-6.0.4 tqdm-4.45.0 twilio-6.26.3 typing-extensions-3.7.4.3 tzlocal-2.1 ujson-2.0.3 urllib3-1.25.10 uvloop-0.14.0 wcwidth-0.2.5 webexteamssdk-1.3 websockets-8.1 werkzeug-1.0.1 wrapt-1.12.1 yarl-1.5.1 zipp-3.1.0
-----> Discovering process types
   Procfile declares types -> (none)
-----> Compressing...
 !     Compiled slug size: 580.3M is too large (max is 500M).
 !     See: http://devcenter.heroku.com/articles/slug-size
 !     Push failed

I have included a .gitignore, .dockerignore and .slugignore to reduce size of docker image and slug. Yet slug size isn’t reducing. What could have caused this and how do I tackle this ? Please find my Dockerfiles below.

1. Rasa

FROM rasa/rasa:latest

COPY . /app

WORKDIR /app

RUN  rasa train

WORKDIR /app

# Copy any additional custom requirements, if necessary (uncomment next line)
COPY actions/requirements-actions.txt ./

# Change back to root user to install dependencies
USER root

RUN apt-get update rm -rf /var/lib/apt/lists/* &&\
    apt-get -y install gcc --no-install-recommends &&\
    pip3 install -r requirements-actions.txt
# Install extra requirements for actions code, if necessary (uncomment next line)

WORKDIR /app

# By best practices, don't run the code with root user

RUN chmod -R 777 /app/scripts/*

ENTRYPOINT []

CMD  bash /app/scripts/start_services.sh

2. Rasa-SDK

FROM rasa/rasa-sdk:latest


# Use subdirectory as working directory
WORKDIR /app

# Copy any additional custom requirements, if necessary (uncomment next line)
COPY requirements-actions.txt ./

# Change back to root user to install dependencies
USER root

RUN apt-get update rm-rf/var/lib/apt/lists/* &&\
    apt-get -y install gcc --no-install-recommends &&\
    pip3 install -r requirements-actions.txt
# Install extra requirements for actions code, if necessary (uncomment next line)

WORKDIR /app

# By best practices, don't run the code with root user

ENTRYPOINT []

CMD python3 -m rasa-sdk --actions actions -p $PORT

USER 1001

3. heroku.yml

build:
  docker:
    web: Dockerfile
run:
  web: run python -m rasa_core.run --enable_api --core models -c rest --endpoints config/endpoints.yml --credentials config/credentials.yml -u current/
  actions: echo test

I am trying to have Continuous integrations (Automatic deploys) from my github repo.

Is there a rasa-slim or rasa-lite or rasa-bare docker image available ? The rasa docker image seems to be too big. How can I reduce the slugsize further to deploy the bot ?

Please help

Regards,

Sanjay

Hey @_sanjay_r unfortunately you’re already using the smallest available docker image - if you take a look at all the others on docker hub they’re even bigger. Is there maybe a different way of deploying things with heroku, e.g. according to the docker documentation?