Hi all, I’ll be experimenting with interactive widgets to help explain our tools a bit better. In particular I am interesting in this that might make it easier to understand our new DIET architecture. So I started experimenting with interactive widgets. You can see a preview here; [45] The widget will highlight which hyperparameters you can set on that particular part of the architecture. You can play with this interactive widget here and I would love to get some feedback on it. Please let me know what you think!

This is really cool! The first thing I notice is that the parameter visibility is activated on hovering the shape, but deactivated when hovering on the word within the shape—I wonder if its possible to fix that—after this it seemed to me I might want to “pin” certain info on the screen and not have it disappear when I hover away: perhaps a something as simple as a checkbox on each shape, would allow me to select the shapes corresponding to the nodes I’m interested in, and then see a summary of all of their parameters listed in sequence…

Oh that’s definately something that is fixable, but it requires some manual labor. The svg box is currently behind the text. I can put another one that is see-through on top. The current version was made quickly to get it demo-ready and no further before I got community feedback. I like your idea of perhaps “clicking” the box to maintain the state. That way you can copy/paste the settings that we list below. Will keep it in mind :slight_smile: There are two main concerns still on my side; Will this really help folks? Or is time better spent making other content. Where to put it. This may not be great for the docs page.

Will this really help folks? Or is time better spent making other content. This is of course impossible to know a priori, but maybe we can put a still-relatively-minimal version of this in place and get some feedback? Where to put it. This may not be great for the docs page. I agree in so far as it suggests the kind of interactive illustration that I would love to eventually see on the main site, because it’s so engaging. But docs might still be a good place to test it, as an approach

Hi Vincent, I also think this widget is awesome, I would also support the idea of having this widget in the docs, particularly different versions of it (like the three in your benchmark video) so users can see the configuration options and their effect interactively. Another argument in favour of having it in the docs, is that it will probably highlight the differences when using different entity extractors and why some parameters do not really apply to the DIET classifier (like bias and pattern). I think this is worth investing time especially since it’s the “default” classifier now.

hi, i have some question about the role of feed foward in DIET

@setopaisen AskMeAnything[tm]. Just to check though, have you seen the videos here ?

already seen the videos :slight_smile:, i think the answer is here but i cant understand in this part [ask]

I’ll try to emphesize a few points here, but could you be more specific about what part you don’t understand? You’re confused about the sparsity? The main idea here is that from our research it seems like most of the weights can be removed from a dense layer. A lot of these weights do not make the model much better. But instead of training a model and then remove the weights in hindsight, we just start with sparse feed forward layers from the get-go. [image] It’s those sparse feed forward layers that we use inside of DIET. Note that the base setting is to remove 80% of the weights but this is a hyperparameter that you can tune.

yes, i’m confused about sparsity and now I understand about this part “do we really need all those connection ?” Thanks about your detailed explaination. Actually if you don’t mind, there’s another part that i would like to ask again, it’s about how does embedding layer that go out from CLS Transformers and intent looks like. But it’s ok I’ll wait for the white board algorithm series :slight_smile:

If you could check the youtube videos first, then if there’s still some extra questions you can start a new question here on the forum and tag me. I’ll gladly answer any question you might have on DIET there.

Interactive Widgets to explain DIET

Rasa Open Source

duongkstn (Đào Nguyên Dương ) May 22, 2020, 8:41am 13

great work !! thanks

Topic		Replies	Views
SHARE_HIDDEN_LAYERS confusion Rasa Open Source	0	275	June 14, 2022
DIET Architecture - Individual Token Pathway Feedback on Rasa Open Source	4	969	November 14, 2021
Hyperparameters for Pipeline Components Rasa Open Source	2	418	October 2, 2021
Question about DIET classifier implementation details - Are featurizers trained? (and others) Rasa Open Source	1	373	May 4, 2023
DIETClassifier and Other Language Rasa Open Source	2	550	August 18, 2020

Interactive Widgets to explain DIET

Related topics