Hi @jonathanpwheat!
Hmm… 
Well, the CRFEntityExtractor is able to use the dense features from the featurizers for entity extraction, so I thought maybe DIET could do that too.
Anyways, one more last thing to try 
from my side, keep the SpacyFeaturizer and put CRFEntityExtractor in the config as well (before DIET).
Please try it out, let’s see if it works 