Add int4 & uint4 types to MigraphX

[lakhinderwalia]

Add int4 & uint4 types to MigraphX

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 37.50000% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/shape.cpp	28.57%	5 Missing :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3378      +/-   ##
===========================================
- Coverage    92.27%   92.25%   -0.02%     
===========================================
  Files          500      500              
  Lines        20057    20061       +4     
===========================================
  Hits         18506    18506              
- Misses        1551     1555       +4     

Files with missing lines	Coverage Δ
src/api/api.cpp	73.30% <ø> (ø)
src/include/migraphx/shape.hpp	88.24% <100.00%> (ø)
src/shape.cpp	95.58% <28.57%> (-1.07%)	:arrow_down:

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pfultz2]

In addition, we need to add unit tests for this type.

check_shapes needs to be updated to throw an error on these types with a flag to allow it for operators like unpack_int4, similar to the dynamic_allowed flag.

We also need a way to automatically insert the unpack_int4 for arguments in the onnx parser so we avoid passing it to unssuported operators(which is everything except unpack_int4).

I wont be around the next couple of weeks to help design or review this, so this will have to wait until september to tackle this further.

[pfultz2]

Also, we need update the bytes calculation to be the correct amount. We also need to throw when calling index calculations.

[migraphx-bot]

Test	Batch	Rate new 45e039	Rate old ae2b02	Diff	Compare
torchvision-resnet50	64	3,238.24	3,232.34	0.18%	:white_check_mark:
torchvision-resnet50_fp16	64	6,883.75	6,875.78	0.12%	:white_check_mark:
torchvision-densenet121	32	2,427.11	2,429.76	-0.11%	:white_check_mark:
torchvision-densenet121_fp16	32	4,067.56	4,068.80	-0.03%	:white_check_mark:
torchvision-inceptionv3	32	1,633.02	1,636.23	-0.20%	:white_check_mark:
torchvision-inceptionv3_fp16	32	2,742.59	2,744.68	-0.08%	:white_check_mark:
cadene-inceptionv4	16	771.06	771.82	-0.10%	:white_check_mark:
cadene-resnext64x4	16	803.26	802.64	0.08%	:white_check_mark:
slim-mobilenet	64	7,435.39	7,438.28	-0.04%	:white_check_mark:
slim-nasnetalarge	64	207.39	207.40	-0.01%	:white_check_mark:
slim-resnet50v2	64	3,340.59	3,328.69	0.36%	:white_check_mark:
bert-mrpc-onnx	8	1,147.88	1,149.05	-0.10%	:white_check_mark:
bert-mrpc-tf	1	309.56	308.98	0.19%	:white_check_mark:
pytorch-examples-wlang-gru	1	433.92	416.23	4.25%	:high_brightness:
pytorch-examples-wlang-lstm	1	455.59	374.95	21.51%	:high_brightness:
torchvision-resnet50_1	1	807.35	799.37	1.00%	:white_check_mark:
cadene-dpn92_1	1	396.39	432.81	-8.41%	:red_circle:
cadene-resnext101_1	1	376.53	378.19	-0.44%	:white_check_mark:
onnx-taau-downsample	1	344.18	345.37	-0.34%	:white_check_mark:
dlrm-criteoterabyte	1	35.03	35.05	-0.05%	:white_check_mark:
dlrm-criteoterabyte_fp16	1	57.32	57.40	-0.14%	:white_check_mark:
agentmodel	1	7,860.89	9,772.50	-19.56%	:red_circle:
unet_fp16	2	57.76	57.79	-0.05%	:white_check_mark:
resnet50v1_fp16	1	931.66	939.67	-0.85%	:white_check_mark:
resnet50v1_int8	1	981.22	929.28	5.59%	:high_brightness:
bert_base_cased_fp16	64	1,141.98	1,142.07	-0.01%	:white_check_mark:
bert_large_uncased_fp16	32	351.81	351.79	0.01%	:white_check_mark:
bert_large_fp16	1	208.66	209.75	-0.52%	:white_check_mark:
distilgpt2_fp16	16	2,152.42	2,151.98	0.02%	:white_check_mark:
yolov5s	1	504.54	506.15	-0.32%	:white_check_mark:
tinyllama	1	43.46	43.34	0.26%	:white_check_mark:
vicuna-fastchat	1	170.39	169.35	0.61%	:white_check_mark:
whisper-tiny-encoder	1	410.36	411.82	-0.36%	:white_check_mark:
whisper-tiny-decoder	1	419.44	429.16	-2.26%	:white_check_mark:

This build is not recommended to merge :red_circle:

[migraphx-bot]

     :white_check_mark: bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     :white_check_mark: bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     :white_check_mark: pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     :white_check_mark: pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     :white_check_mark: torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     :white_check_mark: cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     :white_check_mark: cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     :white_check_mark: dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     :white_check_mark: agentmodel: PASSED: MIGraphX meets tolerance

     :white_check_mark: unet: PASSED: MIGraphX meets tolerance

     :white_check_mark: resnet50v1: PASSED: MIGraphX meets tolerance

     :white_check_mark: bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

:red_circle:bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     :white_check_mark: bert_large: PASSED: MIGraphX meets tolerance

     :white_check_mark: yolov5s: PASSED: MIGraphX meets tolerance

     :white_check_mark: tinyllama: PASSED: MIGraphX meets tolerance

     :white_check_mark: vicuna-fastchat: PASSED: MIGraphX meets tolerance

     :white_check_mark: whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     :white_check_mark: whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     :white_check_mark: distilgpt2_fp16: PASSED: MIGraphX meets tolerance