cpu_convert enable u4/i4 ->f32,bf16,i8,u8

[xipingyan]

Details:

• cpu_convert enable u4/i4 -> f32/bf16/i8/u8

Tickets:

• 124834
• 139282
• 140112

[xipingyan]

@v-Golubev @dmitry-gorokhov @ceciliapeng2011 could you help review? I am not sure if I need to add unit test. What's your idea? @dmitry-gorokhov @v-Golubev

[v-Golubev]

@v-Golubev @dmitry-gorokhov @ceciliapeng2011 could you help review? I am not sure if I need to add unit test. What's your idea? @dmitry-gorokhov @v-Golubev

Regarding tests: I think we can add new instances to conversion.cpp.

[xipingyan]

@v-Golubev @dmitry-gorokhov @ceciliapeng2011 could you help review? I am not sure if I need to add unit test. What's your idea? @dmitry-gorokhov @v-Golubev

Regarding tests: I think we can add new instances to conversion.cpp.

I can't add my test to here direclty, because this test takes parameter as input, but OV model don't support u4/i4 as input precision. https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_cpu/src/plugin.cpp#L258

Maybe I need to construct a subgraph to test it.

[maxnick]

@ceciliapeng2011 , could you please review?

[xipingyan]

@v-Golubev @dmitry-gorokhov @ceciliapeng2011 could you help review? I am not sure if I need to add unit test. What's your idea? @dmitry-gorokhov @v-Golubev

Regarding tests: I think we can add new instances to conversion.cpp.

I can't add my test to here direclty, because this test takes parameter as input, but OV model don't support u4/i4 as input precision. https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_cpu/src/plugin.cpp#L258

Maybe I need to construct a subgraph to test it.

@praasz Because I update https://github.com/openvinotoolkit/openvino/pull/24297/files#diff-308b90982b390cd43d0e0e3db21f9462c8f4847639dc054d24118a1b333cb5c6R243, so I can add test now.

[dmitry-gorokhov]

@Wovchena Please take a look on BA updates

[xipingyan]

Add my verification codes:

1: Construct model (python code)

def model():
    input = opset.parameter([128], ov.Type.i4, name='in')

    cvt = opset.convert(input, ov.Type.f32)
    Result = opset.result(cvt, name='cvt')
    return ov.Model([Result], [input], 'MyModel')

# Ov model and inference
core = ov.Core()
m = model()
ov.save_model("convert_model.xml", m)

2: Generate bin file (python code)

def save_bin(shape, out_fn):
    z = np.zeros(shape, dtype=np.uint8)
    with open(out_fn,"wb") as file:
        file.write(z.tobytes())
save_bin((64), '128_u4.bin')

3: benchmark test (bash)

benchmark_app=../bin/intel64/Release/benchmark_app
model='./convert_model.xml'
# input random data
$benchmark_app -m $model -niter 1 
# input bin file
u4_input=in:128_u4.bin
$benchmark_app -m $model -niter 1 -i $u4_input

[xipingyan]

Thank you! When you posted your verification process, I realized that it could be a good idea to extend tests with your cases: https://github.com/openvinotoolkit/openvino/blob/master/tests/samples_tests/smoke_tests/test_benchmark_app.py. That would also verify CPU plugin work in the end to end scenario. This is optional, I'm not going to require that.

hi @Wovchena , I try to add a 4bit pytest, but I need to push model and test data(bin file). where to push? storage.openvinotoolkit.org? I can't download test data from it now. Or I can construct a model and we only test random data input, is it OK?

[Wovchena]

I was thinking about the model from your snippet. You can generate this model during the test execution, so no downloading is required. The same is for test data, just fix the seed.

[dmitry-gorokhov]

I think we can procceed with the test in separate PR. Merging that one. Thanks!