Hey all,
I want to still use SpacyTokenizer (due to lemma and token attributes), but returning tokens as ‘20’, ‘/’ and ‘03’ for a string like ‘20/03’. Problem should be solved adding a special case about considering ‘/’ as a token itself.
As I don’t know how to operate well in the spacy lib components, before doing a deep dive study on the spacy library I wonder if anyone already did this modification or something similar that would help with this.
Any suggestion?