Found it, on one of the other rasa pages… so too well hidden.
I have to change from user 1001 to root first, and then go back. The final Dockerfile is:
FROM rasa/rasa:1.10.14-full
# Change back to root user to install dependencies
USER root
RUN python -m spacy download nl_core_news_sm
RUN python -m spacy link nl_core_news_sm nl
# Switch back to non-root to run code
USER 1001