[one-cmds] define optimizations

[seanshpark]

There was an old issue about one-optimize group options, but as I remember without concrete conclusion about what is with what.

Now, it's time to define them.

[seanshpark]

#9190 is already fired but we need to (1) define group of optimization and (2) switchs of pass for each group.

[YongseopKim]

Related issue: https://github.com/Samsung/ONE/issues/5784

[YongseopKim]

Very sorry for late.

Why did I do like https://github.com/Samsung/ONE/pull/9190?

• I thought that almost options are okay
• So I think it's okay to turn on all options
• If so, the default option supports all true options
• and then, let's support for specific models such as tf, tflite, onnx if users want

[YongseopKim]

[one-cmds] define optimizations There was an old issue about one-optimize group options, but as I remember without concrete conclusion about what is with what.

Is it for just one-optimize? or all passes(eg. from one-import, ... to one-optimize)?

(1) define group of optimization

group means by model type(tf, tflite, ...)? I guess so.

(2) switchs of pass for each group

I first planned like that

• Prepare files for each optimization group: ODEFAULT_GROUP_A, ODEFAULT_GROUP_B
• The file contains like https://github.com/Samsung/ONE/pull/9190
• Pass the parameter of one-quantize with DEFAULT_GROUP_A
• So, it doesn't switchs of pass for each group

However, now I fix my idea after discussion with @jyoungyun on 5/27. The maintain for the files can be very expensive. So now I think the optimization group can be done by one-quantize.

[seanshpark]

Is it for just one-optimize? or all passes?

I don't understand the question. one-import just imports source model (tf, tflite, onnx, ...) to circle model. I don't know what related you want to ask with one-optimize.

group means by model type(tf, tflite, ...)? I guess so.

Maybe so, but not from model source type but optimization purpose. This is currently fuzzy for me too.

ODEFAULT_GROUP_A, ODEFAULT_GROUP_B

I'm against using word DEFAULT and adding suffix to word DEFAULT makes more confusion.

Pass the parameter of one-quantize with DEFAULT_GROUP_A So now I think the optimization group can be done by one-quantize.

Why does one-quantize come here? Please explain more about this.

So, it doesn't switchs of pass for each group

This too. I can't catch this.

[seanshpark]

I thought that almost options are okay So I think it's okay to turn on all options If so, the default option supports all true options

ONNX related NCHW to NHWC and maybe (not sure) with transpose, reshape, may give unexpected results from TF models. This is the main issue from your draft.

And as I wrote above, I'm against using word DEFAULT.

[seanshpark]

The word group I mentioned needs to be changed to another word for approrpriate purpose.

[YongseopKim]

I don't understand the question. one-import just imports source model (tf, tflite, onnx, ...) to circle model. I don't know what related you want to ask with one-optimize.

I asked this because of https://github.com/Samsung/ONE/pull/9190#issuecomment-1139398914 . So I thought optimization could be included from one-import to one-quantizaiton if possible. (Of course, I could misunderstand 😢 )

[seanshpark]

I asked this because of https://github.com/Samsung/ONE/pull/9190#issuecomment-1139398914 .

@mhs4670go , can you clarify what you mean in the comment? and about So I thought optimization could be included from one-import to one-quantizaiton if possible. ?

[mhs4670go]

@YongseopKim The old issue that @seanshpark is referring to is https://github.com/Samsung/ONE/issues/5784. And, this is for one-optimize and technically, it is for circle2circle optimization pass.

"optimization" will be able to be applied to other one-cmds tools as well as one-optimize.

@YongseopKim 's thought is from this. "optimization" would be able to have broader meaning. For example, we could need some optimization or preprocessing in import step for domain specific import. But, we've never had a discussion on this. This is a feature for future.

[YongseopKim]

@YongseopKim 's thought is from this. "optimization" would be able to have broader meaning. For example, we could need some optimization or preprocessing in import step for domain specific import. But, we've never had a discussion on this. This is a feature for future.

I thought there was a consensus on it. So, now I don't think of optimization as a broader meaning. I focus on optimization for circle-circle, one-optimize.

[YongseopKim]

Maybe so, but not from model source type but optimization purpose. This is currently fuzzy for me too. I'm against using word DEFAULT and adding suffix to word DEFAULT makes more confusion. ONNX related NCHW to NHWC and maybe (not sure) with transpose, reshape, may give unexpected results from TF models. This is the main issue from your draft.

While I'm reading these comments, I realize that I'm using mixed meaning of default. I use it as 'basic options(meaning as it is default)' and 'optimized options as default'.

[YongseopKim]

And this issue is for defining an optimization list for each group(eg. onnx_with_transpose, ..., not yet defined).

'basic options(meaning as it is the default)' and 'optimized options as default' should be handled by different issues.

[YongseopKim]

Before suggesting my idea for Optimization Options, let's define when the optimization option groups(?) are necessary.

• The word group is a temporary word.
• Feel free to comment as you think weird.

When?

1. User doesn't have any cfg file so that uses onecc init with its own model. ONE wants to support possible optimization options. ONE knows the only model type. So onecc init wants to fill an optimization options group for the model type.
2. User has a cfg file and can make the final compiled output by the backend. The user wants to optimize it simply/easily. So user uses onecc optimize with some option groups (or by fixing cfg file).
3. User has a cfg file and is good at ONE. The user wants to optimize at most. The user knows the main operations of the model and other compiled options.

I suggest

• for No.1: one-optimize would have each unacceptable list for each model type and then support it. (Of course, this can be supported by some scripts or programs.)
• for No.2: let's introduce O2 expanding O1. O2 could include as much as possible as an option for all models. This can be implemented in circle2circle same as O1's case.
• for No.3: let's introduce very specific option groups like OptionsForTransformerModel, OptionsForRNNModel, ... I think it's a future item.

I'd like to get your opinions on the suggestion.

[seanshpark]

for the beginning, +1 to No.1