anipose
anipose copied to clipboard
lambdaloop
anipose analyze extremely slow
Hello I am new to anipose and not very used to python yet. I just installed anipose following the guidelines in https://anipose.readthedocs.io/en/latest/installation.html. I am trying to run the tutorial for the hand-demo dataset and the video analysis is running impossibly slow... my it/s is around 1.5 it/s... more than 30 minuites per video. As I see in the Google Slides presentation, it seems that it should be faster (14 it/s). Also, the output looks rather different compared to Google Slides. Quoted below is the output of the anipose analize from my machine
$ anipose analyze Analyzing videos... OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids. OMP: Info #213: KMP_AFFINITY: cpuid leaf 11 not supported - decoding legacy APIC ids. OMP: Info #149: KMP_AFFINITY: Affinity capable, using global cpuid info OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-23 OMP: Info #156: KMP_AFFINITY: 24 available OS procs OMP: Info #157: KMP_AFFINITY: Uniform topology OMP: Info #159: KMP_AFFINITY: 1 packages x 1 cores/pkg x 24 threads/core (1 total cores) OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map: OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 thread 0 OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 thread 1 OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 thread 2 OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 thread 3 OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 thread 4 OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 thread 5 OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 thread 8 OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 thread 9 OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 thread 10 OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 thread 11 OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 thread 12 OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 thread 13 OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 thread 16 OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 thread 17 OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 thread 18 OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 thread 19 OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 0 thread 20 OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 0 thread 21 OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 0 thread 24 OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 0 thread 25 OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 0 thread 26 OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 0 thread 27 OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 0 thread 28 OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 0 thread 29 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5569 thread 0 bound to OS proc set 0 /home/ozge/Anipose Tutorial/hand-demo-unfilled/2019-08-02/videos-raw/2019-08-02-vid01-camA.MOV /home/ozge/Anipose Tutorial/hand-demo-unfilled/2019-08-02/videos-raw/2019-08-02-vid01-camB.MOV /home/ozge/Anipose Tutorial/hand-demo-unfilled/2019-08-02/videos-raw/2019-08-02-vid01-camC.MOV /home/ozge/Anipose Tutorial/hand-demo-unfilled/2019-08-02/videos-raw/2019-08-02-vid02-camA.MOV /home/ozge/Anipose Tutorial/hand-demo-unfilled/2019-08-02/videos-raw/2019-08-02-vid02-camB.MOV WARNING:tensorflow:From /home/ozge/anaconda3/envs/gregorioEnv/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /home/ozge/anaconda3/envs/gregorioEnv/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /home/ozge/anaconda3/envs/gregorioEnv/lib/python3.7/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. WARNING:tensorflow:From /home/ozge/anaconda3/envs/gregorioEnv/lib/python3.7/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. 0%| | 0/4395 [00:00<?, ?it/s]OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5602 thread 1 bound to OS proc set 1 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5664 thread 3 bound to OS proc set 3 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5667 thread 6 bound to OS proc set 6 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5669 thread 8 bound to OS proc set 8 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5665 thread 4 bound to OS proc set 4 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5668 thread 7 bound to OS proc set 7 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5671 thread 10 bound to OS proc set 10 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5663 thread 2 bound to OS proc set 2 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5670 thread 9 bound to OS proc set 9 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5666 thread 5 bound to OS proc set 5 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5672 thread 11 bound to OS proc set 11 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5673 thread 12 bound to OS proc set 12 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5674 thread 13 bound to OS proc set 13 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5676 thread 14 bound to OS proc set 14 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5677 thread 15 bound to OS proc set 15 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5678 thread 16 bound to OS proc set 16 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5679 thread 17 bound to OS proc set 17 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5680 thread 18 bound to OS proc set 18 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5681 thread 19 bound to OS proc set 19 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5682 thread 20 bound to OS proc set 20 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5683 thread 21 bound to OS proc set 21 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5684 thread 22 bound to OS proc set 22 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5685 thread 23 bound to OS proc set 23 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5686 thread 24 bound to OS proc set 0 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5603 thread 25 bound to OS proc set 1 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5688 thread 27 bound to OS proc set 3 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5687 thread 26 bound to OS proc set 2 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5689 thread 28 bound to OS proc set 4 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5690 thread 29 bound to OS proc set 5 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5691 thread 30 bound to OS proc set 6 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5692 thread 31 bound to OS proc set 7 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5694 thread 33 bound to OS proc set 9 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5695 thread 34 bound to OS proc set 10 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5693 thread 32 bound to OS proc set 8 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5696 thread 35 bound to OS proc set 11 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5697 thread 36 bound to OS proc set 12 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5698 thread 37 bound to OS proc set 13 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5700 thread 39 bound to OS proc set 15 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5701 thread 40 bound to OS proc set 16 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5699 thread 38 bound to OS proc set 14 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5702 thread 41 bound to OS proc set 17 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5703 thread 42 bound to OS proc set 18 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5704 thread 43 bound to OS proc set 19 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5706 thread 45 bound to OS proc set 21 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5707 thread 46 bound to OS proc set 22 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5705 thread 44 bound to OS proc set 20 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5709 thread 48 bound to OS proc set 0 OMP: Info #250: KMP_AFFINITY: pid 5569 tid 5708 thread 47 bound to OS proc set 23 55%|█████████████████████▎ | 2408/4395 [25:34<20:31, 1.61it/s]
It looks to me like something is wrong but I am completely lost. Did I make a mistake during the installation? ... please help!!
In case it is useful, I am using Ubuntu 18.04 with an NVIDIA GV102 gpu where I run DLC without any issues.
Hello, it seems that it's not using the GPU, but I'm not sure exactly why.
Do you have DeepLabCut and anipose installed in the same environment? When you run DeepLabCut on any one of the videos, do they run at a fast speed?
DeepLabCut was already installed in this machine and working fine and it has its own enviroment. When I installed anipose I created a new environment for it but I did not re-install DLC.
If I analyze the videos in the DLC environment it goes much faster (28 it/s)
I had the same problem but when I installed anipose by following the instruction, I changed "tensorflow=1.13.1" to "tensorflow-gpu=1.13.1" and now it is using GPU.
I am having the same problem. It’s not using my GPU for “anipose analyze”. Is there any solution please ?
Hello, I encountered the same problem. The speed is very slow and the GPU is not used. Have you solved it?
