Trying to escape unicode characters and not language characters

Hi, I am trying to export a dataframe with hindi sentences, and I have added following line to escape emojis in sentences or some illegal characters.

intent_df = intent_df.applymap(lambda x: x.encode('unicode_escape').
                                   decode('utf-8') if isinstance(x, str) else x)

but this also removes hindi characters. And I am sure this same thing will happen with other languages too. So, can anyone help me out on how to just escape emojis and some unwanted symbols?

Does the data 100% have to go into Excel? We talked a bit about this on slack and I’m wondering if this might be a Windows/Excel thing as opposed to a unicode thing.

Yeah I am exporting and importing data to and fro yml and excel since LSP’s don’t accept yml an. So, I want to make sure that data is properly exported and imported in the format that Rasa accepts

If you cast it to base csv files and you inspect the text, does the issue persist there as well?

Same for when you drop .(encode('unicode_escape')?

I did tried to export to csv but nlu and domain has commas and tabs so TSV is also not possible