I’ll be experimenting with interactive widgets to help explain our tools a bit better. In particular I am interesting in this that might make it easier to understand our new DIET architecture. So I started experimenting with interactive widgets. You can see a preview here;
The widget will highlight which hyperparameters you can set on that particular part of the architecture. You can play with this interactive widget here and I would love to get some feedback on it. Please let me know what you think!
This is really cool! The first thing I notice is that the parameter visibility is activated on hovering the shape, but deactivated when hovering on the word within the shape—I wonder if its possible to fix that—after this it seemed to me I might want to “pin” certain info on the screen and not have it disappear when I hover away: perhaps a something as simple as a checkbox on each shape, would allow me to select the shapes corresponding to the nodes I’m interested in, and then see a summary of all of their parameters listed in sequence…
Oh that’s definately something that is fixable, but it requires some manual labor. The svg box is currently behind the text. I can put another one that is see-through on top. The current version was made quickly to get it demo-ready and no further before I got community feedback.
I like your idea of perhaps “clicking” the box to maintain the state. That way you can copy/paste the settings that we list below. Will keep it in mind
There are two main concerns still on my side;
Will this really help folks? Or is time better spent making other content.
Where to put it. This may not be great for the docs page.
Will this really help folks? Or is time better spent making other content.
This is of course impossible to know a priori, but maybe we can put a still-relatively-minimal version of this in place and get some feedback?
Where to put it. This may not be great for the docs page.
I agree in so far as it suggests the kind of interactive illustration that I would love to eventually see on the main site, because it’s so engaging. But docs might still be a good place to test it, as an approach
Hi Vincent, I also think this widget is awesome, I would also support the idea of having this widget in the docs, particularly different versions of it (like the three in your benchmark video) so users can see the configuration options and their effect interactively. Another argument in favour of having it in the docs, is that it will probably highlight the differences when using different entity extractors and why some parameters do not really apply to the DIET classifier (like bias and pattern). I think this is worth investing time especially since it’s the “default” classifier now.
I’ll try to emphesize a few points here, but could you be more specific about what part you don’t understand? You’re confused about the sparsity?
The main idea here is that from our research it seems like most of the weights can be removed from a dense layer. A lot of these weights do not make the model much better. But instead of training a model and then remove the weights in hindsight, we just start with sparse feed forward layers from the get-go.
It’s those sparse feed forward layers that we use inside of DIET. Note that the base setting is to remove 80% of the weights but this is a hyperparameter that you can tune.
yes, i’m confused about sparsity and now I understand about this part “do we really need all those connection ?”
Thanks about your detailed explaination.
Actually if you don’t mind, there’s another part that i would like to ask again, it’s about how does embedding layer that go out from CLS Transformers and intent looks like. But it’s ok I’ll wait for the white board algorithm series
If you could check the youtube videos first, then if there’s still some extra questions you can start a new question here on the forum and tag me. I’ll gladly answer any question you might have on DIET there.
, i think the answer is here but i cant understand in this part 
