I finally figured it out , rasa core is a part of Rasa X , so you do not need to install it if you already have Rasa X.
To query the model on say rest channel , you need to add rest in your credentials.yml file, so it look like this:
rasa:
url: ${RASA_X_HOST}/api
rest:
and then you can query your models on