ONE-vscode
ONE-vscode copied to clipboard
Samsung
[MPQEditor] Create or edit mixed precision quantization json file.
This commit allows to create or edit mixed precision quantization json file. To use the tool the following actions are suggested:
- right-click in explorer any circle model
- context menu should contain "Create MPQ json" - click it
- provide name for new .json
- after creation of new json we may edit it
Related: #1491 ONE-vscode-DCO-1.0-Signed-off-by: s.malakhov [email protected]
Q) How visq info is created? Is it assumed to be generated before? If so, it would be better to give instruction on how to make visq info (e.g., making a small question mark icon near the "Load visq info" button). BTW, is it possible to create visq info in ONE-vscode?
(optional) I think it is better to show the model graph by default when the tab is first created. I guess most users will want to select layers in the graph with visq info.
Q) How visq info is created? Is it assumed to be generated before?
Yes, for this draft it was generated before.
If so, it would be better to give instruction on how to make visq info (e.g., making a small question mark icon near the "Load visq info" button)
ok.
BTW, is it possible to create visq info in ONE-vscode?
I'll investigate it.
(optional) I think it is better to show the model graph by default when the tab is first created. I guess most users will want to select layers in the graph with visq info.
Ok.
I'll be back with reworked draft.
BTW, is it possible to create visq info in ONE-vscode?
I've tried to introduce visq data computation into MPQEditor and it occured to be heavy: input for float model input for quantized model input for h5 data ouput for MPEIR mode ouput for MSE mode ... to be continued
IMHO for visq component we may try to create additional command in ONE-vscode.
@jinevening
May be we could do another item in context menu for circle file like "Create VISQ data" ?
@jinevening @seanshpark
Please see docs/MPQEditor.md for readme.
For now model-graph-view is opened by default.
And help btn is also introduced into visq_path input.
Thanks for video attach at https://github.com/Samsung/ONE-vscode/pull/1505#issuecomment-1434501372 :)
Q1) what happens if wrong json (not visq info content) is selected in visq info ? Q2) what happens if wrong json (visq info content but for different model) is selected in visq info ? Q3) there seems to be no way to clear loaded visq info. is there any particular reason for this?
model.json filename doesn't seem to be appropriate. we can't distinguish by just .json. how about use model.blabla.json ? @jinevening , do you have any good word for blabla ?
1/ what is this icon?
2/ may use "Edit mixed precision quantization json file" ? for M.P.Q. or just Edit MPQ json file ?
About GUI
Q1) what are default parameters? what happens if we switch between uint8 and int16 ?
Q2) same question for Granularity
Q3) from select node with MPQEditor view and from both graph view; select and unselect from both view. selection information seems strange. look at X button.
With right click context menu > open with > text editor 1/ it would be better to save json file with formatted not in single line 2/ edit-save each view should sync. please check if the content are same.
With Model Graph is shown
1/ after graph is visible, close window by X in the TAB
-> check button doesn't work afterwards
2/ Open visq.json file -> graph is changed to visquv -> close graph view -> open graph view again
-> view is not visquv
I got this popup. cannot understand about the message
In ~README~ docs/MPQEditor.md , please describe GUI actions for each controls(GUI components) and what it does. Purpose of this description is (1) to make you develop the GUI part is working correctly and (2) let users understand the actions what this feature provides.
And please check with interactions with controls that you assume don't matter each other but can interact. This test is to let you handle exceptions what user can do beyond your user scenario. I think this is know as monkey test.
About GUI no.2
Save MPQEditor json file -> close all windows -> do it again
- json file name is provided same as the first place --> maybe add number prefix like
(1)for the name if exist?
Save MPQEditor json file -> close all windows -> open json file
- plain json text editor is opened --> users will expect to see MPQEditor view
From https://github.com/Samsung/ONE-vscode/pull/1505#issuecomment-1436918918,
May be we could do another item in context menu for circle file like "Create VISQ data" ?
Looks good to me. "Create visq data" is a different job from MPQ editor. So it may be done in another issue.
From https://github.com/Samsung/ONE-vscode/pull/1505#issuecomment-1437581735,
model.json filename doesn't seem to be appropriate. we can't distinguish by just .json. how about use model.blabla.json ? @jinevening , do you have any good word for blabla ?
For that purpose, model.qconfig.json or model.mpq.json would be ok. Or it may be possible to open the json file and check if it contains proper keys ("default_quantization_dtype" and "default_granularity")
Looks good to me. "Create visq data" is a different job from MPQ editor. So it may be done in another issue.
Ok. I'll create another issue.
For that purpose,
model.qconfig.jsonormodel.mpq.jsonwould be ok. Or it may be possible to open the json file and check if it contains proper keys ("default_quantization_dtype" and "default_granularity")
Ok. IMHO mpq looks good.
1/ what is this icon? 2/ may use "Edit mixed precision quantization json file" ? for M.P.Q. or just
Edit MPQ json file?
- Icon is suggested to be used as general info about the tool
- Text message is changed to "Edit mixed precision quantization json file"
Q1) what are default parameters? what happens if we switch between
uint8andint16? Q2) same question forGranularityQ3) from select node with MPQEditor view and from both graph view; select and unselect from both view. selection information seems strange. look atXbutton.
Q1) and Q2) - default parameters of quantization are used for all layers in the neural network, except for nodes in 'LAYERS' section. So if we switch from uint8 to int16, the whole network will be quantized into 'int16'. As for the Granularity field all convolution layers are quantized using either layer or channel mode of granularity.
Q3) model-graph-view contains all layers of the neural network, while 'channels' section contains only specific nodes.
So if we will check some nodes in 'LAYERS' section we may remove them using X button.
With
Model Graph is shown1/ after graph is visible, close window by X in the TAB -> check button doesn't work afterwards 2/ Open visq.json file -> graph is changed to visquv -> close graph view -> open graph view again -> view is not visquv
It will be fixed.
I got this popup. cannot understand about the message
Sorry. It will be changed to: "visq.json file with layer-wise quantization errors"
About GUI no.2
Save MPQEditor json file -> close all windows -> do it again
* json file name is provided same as the first place --> maybe add number prefix like `(1)` for the name if exist?
This option looks good.
Save MPQEditor json file -> close all windows -> open json file
* plain json text editor is opened --> users will expect to see MPQEditor view
Fixed.
1/ document: ratio is not nice to read. text in image is too large compared to text of doc.
- red arrows look not professional. maybe just add number-circle at the position and add description with text in the document(not in the image)
2/ MPQWorkflow.gif is not visible in preview.
3/ https://github.com/Samsung/ONE-vscode/pull/1505#issuecomment-1437705870
-
1/ it would be better to save json file with formatted not in single linenot applied
4/ with visq path, select a file not a visq json file -> graph loading doesn't work
5/ https://github.com/Samsung/ONE-vscode/pull/1505#discussion_r1112402710 not applied
6/ GUI
- default value settings and layer settings proportion is not good
- like + and x button is too large compared to text of
uint8,int16 - maybe add separation line between default settings and layer settings
- like + and x button is too large compared to text of
-
Model graph is shown->Show model graphor justModel graph - icon of
Edit mixed precision...--> cannot catch what it is. maybe remove. -
Edit mixed precision...is this something like a title? if so, how about enlarge text size.
7/ + and x buttons are disabled when ONE package is not installed. some message would be helpful.
8/ use a quite long model, like inception. in graph view, select bottom node. scroll to top. select also top node (with CTRL key down) --> graph is scrolled to somewhere. it would be better to say in the last selected node.
9/ maybe provide multiple select without CTRL down?
10/ ? help text looks like Serif. other texts are Sanserif. better to use one type.
11/ order of controls in ? X [visq path] (search) --> [visq path] (search) X ?
or ? [visq path] search X
@seanshpark @dayo09 @jinevening thank you for your review.
1/ document: ratio is not nice to read. text in image is too large compared to text of doc.
Fixed.
2/
MPQWorkflow.gifis not visible in preview.
Fixed
* `1/ it would be better to save json file with formatted not in single line` not applie
Fixed.
4/ with
visq path, select a file not a visq json file -> graph loading doesn't work
Fixed.
5/ #1505 (comment) not applied
Seems to be fixed. Default parameters is a disctinct figure.
6/ GUI
Fixed.
7/ + and x buttons are disabled when ONE package is not installed. some message would be helpful.
Fixed. Without installed package there is info baloon with message "you need to install ONE Toolchain fo radding more layers to edit"
8/ use a quite long model, like inception. in graph view, select bottom node. scroll to top. select also top node (with CTRL key down) --> graph is scrolled to somewhere. it would be better to say in the last selected node.
Fixed.
9/ maybe provide multiple select without CTRL down?
But this feature is needed at 'selector' mode too, Are you ok with that?
10/ ? help text looks like Serif. other texts are Sanserif. better to use one type.
All fonts are default for now, Pictures use 'Sansserif; font
11/ order of controls in
? X [visq path] (search)-->[visq path] (search) X ?or? [visq path] search X
Fixed.
4/ not fixed 5/ I'll fix this later 8/ not fixed
9/
But this feature is needed at 'selector' mode too, Are you ok with that?
No, not ok.
10/ not fixed. I'm using English Windows 10 and still shows Serif(maybe Times New Roman)
Q) is first check button for each item used only to select for remove button X?
then why not add X button at each item right most position?
we can replace first check button and the X button.
12/ item layout seems strange depending on the region width
- overlaps and not good
- waste with empty space
4/ not fixed Sorry for misunderstanding. There is an attempt to load invalid 'visq' file in the video attached.
https://user-images.githubusercontent.com/112689352/221483050-f8b6d10a-81a3-43b8-94f9-3005d5a6aa11.mp4
Please note error message on failure in the right-bottom part of the screen after that it reverts view window to selector mode.
@seanshpark
Is it wrong behaviour?
What is the expected behaviour in that case?
Q) is first check button for each item used only to select for remove button
X? then why not addXbutton at each item right most position? we can replace first check button and theXbutton.
Yes. You are right. Currently the first checkbox is only for item removal.
I'll remove the checkbox with the x button on the right side.
8/ not fixed it would be better to say in the last selected node
@seanshpark Seems like to stay in the last selected node we have 2 options:
- disable
_scrollToSelectedeither in constructor or through some kind of message - scrollToSelection scrolls just to last selected element
currently option 2 is implemented.
which one is more preferrable? or may be there is another option, which i don't see right now?
@seanshpark
Q) is first check button for each item used only to select for remove button
X? then why not addXbutton at each item right most position? we can replace first check button and theXbutton
Fixed.
12/ item layout seems strange depending on the region width
Improved.
10/ not fixed. I'm using English Windows 10 and still shows Serif(maybe Times New Roman)
Please correct me if i'm wrong. AFAIK readme.md don't have font settings, at least all other 'readme.md' files n the project don't have font settings.
Page layout (index.html) of all text elements was changed to:
font-family: 'Franklin Gothic Medium', 'Arial Narrow', Arial, sans-serif;
Please note error message on failure in the right-bottom part of the screen after that it reverts view window to selector mode.
Still "loading" and I don't see any error message.
