how to use the net without training

Hi, thank you for this contribution.

It might be a trivial question, but how can I apply FG-Net directly to any image/video?

For my bachlor thesis, I want to evaluate a model based on Action Units. However, I have only little experience so far of running third-party code. I hope someone will help me with that :)

So far, I can assume

• I have to create a json as args and reference my own dataset
• I should download the checkpoints from StyleGAN2?

But now, which file can I use to get action units detected? Is there any file yet or do I have to build my own (based on eval_interpreter.py?) ?

I'm looking forward to some advice :)

Hi,

Thanks for your interests in our work. To get started, you need to download all the checkpoints of StyleGAN2, StyleGAN2 encoder, and our trained models. Please check README for the Google Drive links.

I have added the codes for singe image inference. Specifically, run the following commands with models trained on BP4D/DISFA:

CUDA_VISIBLE_DEVICES=0 python single_image_inference.py --exp experiments/single_image_inference_bp4d.json
CUDA_VISIBLE_DEVICES=1 python single_image_inference.py --exp experiments/single_image_inference_disfa.json

Meanwhile, you can also check our other work (https://github.com/ihp-lab/LibreFace) which is an open-source tool for action unit detection and facial expression analysis.

Let me know if you have any other questions.

First of all, thank you very much for updating your project for my request. It is certainly helpful :) (btw I am also about to use LibreFace :))

However, I sadly could not run it yet:

No CUDA runtime is found, using CUDA_HOME='/homes/usr/miniconda3/envs/fg-net'
Traceback (most recent call last):
  File "single_image_inference.py", line 17, in <module>
...
File "/homes/usr/miniconda3/envs/fg-net/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1606, in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'
IndexError: list index out of range

I checked the versions, but it should be correct as you mentioned in README:

Cuda compilation tools, release 11.3, V11.3.109
Build cuda_11.3.r11.3/compiler.29920130_0

Python 3.7.12
pytorch                   1.10.1          py3.7_cuda11.3_cudnn8.2.0_0    pytorch

I will also ask my supervisor about it, since it might be due to running it as a slurm job. But you could already know this issue and how to fix it.

In any case, thank you again for responding earlier and adding the feature.

Hi SM-Jack,

From my personal feeling, you have a broken install of pytorch or cuda. Maybe print out torch.cuda.is_available() to see if your environment is fine or not.

Otherwise I would suggest ask your supervisor to solve the issue since I am not sure what hardwares you are using.

Best, Yufeng