I recently got access to the GPT-3 beta and I took advantage of the opportunity to test it head-to-head against a simple Rasa chatbot I implemented earlier this year. You can see the results of the comparison, plus some reflections on what GPT-3 means for us, here: https://towardsdatascience.com/gpt-3-vs-rasa-chatbots-8b3041daf91d?source=friends_link&sk=d82fecdad368f19c2ea58637e71d839f
Happy to read people have had similar experiences with GPT-3. My main concern with GPT-3 is the bigotry that it tends to generate though. I’ve noticed that just mentioning a country can lead the generated text to suddenly become somewhat stereotypical.