mariecwhite

Results 33 comments of mariecwhite

Memory use for MobileNetV2 UINT8 2-3x larger than TFlite

Thanks Scott. Git commit hash is 223d02f16. Compiler flags for mobile: ``` "${IREE_COMPILE_PATH}" \ --iree-input-type=tosa \ --iree-hal-target-backends=llvm-cpu \ --iree-llvm-target-triple=aarch64-none-linux-android29 \ --iree-llvm-debug-symbols=false \ --iree-vm-bytecode-module-strip-source-map=true \ --iree-vm-emit-polyglot-zip=false \ "${IREE_MODEL_DIR}/${MODEL_NAME}.mlir" \ --o "${IREE_MODEL_DIR}/llvm-cpu/${MODEL_NAME}.vmfb"...

Legalization of 'tosa.reduce' results in known expand shape op

Hi Natasha, these are newly added models where ResNet-V1-101 is dynamic in batch, width and height. And SSD-MobileNetV2 has dynamic width and height.

Add focus set models to SHARK Tank

Please also add: - [ ] **BertForMaskedLM**: https://huggingface.co/docs/transformers/v4.21.1/en/model_doc/bert#transformers.BertForMaskedLM This model is currently 17x slower than Torch on A100 GPU.

Add focus set models to SHARK Tank

I added two issues to the SHARK repo: Add RNNT: https://github.com/nod-ai/SHARK/issues/329 Add BertForMaskedLM: https://github.com/nod-ai/SHARK/issues/324

[Vulkan] numeric issue for quantized vision models

This was chosen after running MobileBert quantized on a large dataset (Squad v1.1) on x86. We used MLPerf to verify that the final accuracy number was within 0.1% of TFLite....

Run SHARK model suite on IREE post-submit

Yep On Thu, Apr 6, 2023 at 8:06 AM Allie Culp ***@***.***> wrote: > Is this closed? > > — > Reply to this email directly, view it on GitHub...

[Vulkan] numeric issue for quantized vision models

I believe @kuhar was looking into this

Figured out why Bert in Shark tank performs much better than Bert from torchbench

That was a request. I've looked into this and SHARK and TorchDynamo now look the same except for the sequence length. See https://github.com/nod-ai/SHARK/issues/324 for details. I'll close this and we...

Add focus set models to SHARK Tank

These models are no longer high priority. If you are planning to add models, it would be useful to use models in existing benchmarks suites like [MLPerf Inference](https://github.com/mlcommons/inference), [HuggingFace Transformers](https://huggingface.co/docs/transformers/index),...

[CPU] Understand why IREE is 2x slower than TFLite on ViT INT8 on ARM64

Note that the compile command with the patch ends up using cpu features cpu_features `+reserve-x18,+bf16,+crc,+dotprod,+flagm,+fp-armv8,+fullfp16,+fp16fml,+i8mm,+lse,+mte,+pauth,+perfmon,+predres,+spe,+ras,+rcpc,+rdm,+sb,+neon,+ssbs,+sve,+sve2-bitperm,+sve2`