SD/UI: .mlir files saved when explicitly passing --no-save_annotations

[one-lithe-rune]

Don't know if this a bug or intentional but I've noticed I'm getting .mlir files appearing for each flatbuffer in recent builds/commits. This didn't used to be the case.

I still get them even if I set --no-save_annotation at the command line, which I would have thought would have turned them off.

[monorimet]

the --no-save_annotation argument is exclusively used for the proprietary SHARK tuner, which is a separate piece of software that plugs into SHARK python for devs to tune models on. It's looking for a new home.

The .mlir should save to a temporary directory and be deleted after compilation, if they are persistent this is unintended and I can probably fix it quite easily. Can you tell me what the names/lifetimes of the files are like?

[one-lithe-rune]

Can you tell me what the names/lifetimes of the files are like?

Same name as the .vmfb stem, but with _torch_linalg.mlir on the end. Same location as the .vmfb

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        31/10/2023     16:04     2084252845 unet512_1_77_768_512_fp16_tuned_cyberrealistic_v31_vulkan_torch_linalg.mlir
-a----        30/10/2023     09:44       99031537 vae_1_77_512_512_fp16_tuned_cyberrealistic_v31_vulkan_torch_linalg.mlir

[Th3Rom3]

I can confirm this behaviour for the 20231009_984 release. In my case the files are stored in the main folder. I did not realize this wasn't intended behaviour. So far the files are persistent over several days and restarts.

EBWebView
generated_imgs
models
shark_tank
shark_tmp
clip_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan.vmfb
clip_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan_torch_linalg.mlir
clip_1_64_512_512_fp16_v1-5-pruned_cpu.vmfb
clip_1_64_512_512_fp16_v1-5-pruned_cpu_torch_linalg.mlir
clip_1_64_512_512_fp16_v1-5-pruned_vulkan.vmfb
clip_1_64_512_512_fp16_v1-5-pruned_vulkan_torch_linalg.mlir
euler_scale_model_input_1_512_512_cpu_fp16.vmfb
euler_scale_model_input_1_512_512_rocm_fp16.vmfb
euler_scale_model_input_1_512_512_vulkan_fp16.vmfb
euler_step_1_512_512_cpu_fp16.vmfb
euler_step_1_512_512_vulkan_fp16.vmfb
nodai_shark_studio_20231009_984.exe
unet_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan.vmfb
unet_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan_torch_linalg.mlir
unet_1_64_512_512_fp16_v1-5-pruned_cpu.vmfb
unet_1_64_512_512_fp16_v1-5-pruned_cpu_torch_linalg.mlir
unet_1_64_512_512_fp16_v1-5-pruned_vulkan.vmfb
unet_1_64_512_512_fp16_v1-5-pruned_vulkan_torch_linalg.mlir
vae_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan.vmfb
vae_1_64_512_512_fp16_tuned_v1-5-pruned_vulkan_torch_linalg.mlir
vae_1_64_512_512_fp16_v1-5-pruned_cpu.vmfb
vae_1_64_512_512_fp16_v1-5-pruned_cpu_torch_linalg.mlir
vae_1_64_512_512_fp16_v1-5-pruned_vulkan.vmfb
vae_1_64_512_512_fp16_v1-5-pruned_vulkan_torch_linalg.mlir
variants.json

[monorimet]

I have moved the "intermediary" mlir files to be saved in the ./shark_tmp/ directory. https://github.com/nod-ai/SHARK/commit/d051c3a4a7a7cdf645899e05439ccf782fe37b37

Previously, the .mlir was saved in whichever directory the tempfile module uses by default. It seems for windows this registered as $PWD (explaining the behavior seen in this issue) and on Linux it was /tmp/, which causes issues for some users running without root-level permissions.

Ideally we have an option that cleans up this directory at shutdown or after .vmfb compilation, but I'm open to suggestions here.

[one-lithe-rune]

We already do a cleanup of ./shark_tmp for gradio temporary image files from the previous run on studio startup. I'll put up a PR that adds .mlir to that.

After compilation would be cleaner, of course.