sleap
sleap copied to clipboard
talmolab
Problems with Visualizing Model Outputs
Bug description
When attempting to Predict > Visualize Model Outputs... an error occurs.
Expected behaviour
The confidence maps used in training will be plotted/visualized.
Actual behaviour
An error occurs.
Your personal set up
- OS:
Windows 10
- Version(s):
- SLEAP installation method (listed here):
- [ ] Conda from package
- [x] Conda from source
- [ ] pip package
- [ ] M1 Macs
Environment packages
(sleap-retracking) λ conda list
# packages in environment at C:\Users\TalmoLab\miniconda3\envs\sleap-retracking:
#
# Name Version Build Channel
absl-py 1.2.0 pypi_0 pypi
alabaster 0.7.12 pypi_0 pypi
anyio 3.6.1 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
argon2-cffi 21.3.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
atomicwrites 1.4.1 pypi_0 pypi
attrs 21.2.0 pypi_0 pypi
babel 2.10.3 pypi_0 pypi
backcall 0.2.0 pypi_0 pypi
backports-zoneinfo 0.2.1 pypi_0 pypi
beautifulsoup4 4.11.1 pypi_0 pypi
black 21.6b0 pypi_0 pypi
bleach 5.0.1 pypi_0 pypi
ca-certificates 2022.07.19 haa95532_0
cached-property 1.5.2 py_0
cachetools 5.2.0 pypi_0 pypi
cattrs 1.1.1 pypi_0 pypi
certifi 2021.10.8 pypi_0 pypi
cffi 1.15.1 pypi_0 pypi
cfgv 3.3.1 pypi_0 pypi
charset-normalizer 2.1.0 pypi_0 pypi
click 8.0.4 pypi_0 pypi
colorama 0.4.5 pypi_0 pypi
commonmark 0.9.1 pypi_0 pypi
coverage 6.4.2 pypi_0 pypi
cuda-nvcc 11.3.58 hb8d16a4_0 nvidia
cudatoolkit 11.3.1 h59b6b97_2
cudnn 8.2.1 cuda11.3_0
cycler 0.11.0 pypi_0 pypi
debugpy 1.6.2 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
deprecated 1.2.13 pypi_0 pypi
distlib 0.3.5 pypi_0 pypi
docutils 0.17.1 pypi_0 pypi
efficientnet 1.0.0 pypi_0 pypi
entrypoints 0.4 pypi_0 pypi
ffmpeg 4.2.2 he774522_0
filelock 3.7.1 pypi_0 pypi
flatbuffers 2.0 pypi_0 pypi
fonttools 4.34.4 pypi_0 pypi
freetype 2.10.4 hd328e21_0
gast 0.5.3 pypi_0 pypi
geos 3.8.0 h33f27b4_0
gitdb 4.0.9 pypi_0 pypi
gitpython 3.1.27 pypi_0 pypi
google-auth 2.9.1 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
greenlet 1.1.2 pypi_0 pypi
grpcio 1.47.0 pypi_0 pypi
h5py 3.1.0 nompi_py37h19fda09_100 conda-forge
hdf5 1.10.6 h1756f20_1
hdmf 3.3.2 pypi_0 pypi
icc_rt 2019.0.0 h0cc432a_1
icu 69.1 h0e60522_0 conda-forge
identify 2.5.3 pypi_0 pypi
idna 3.3 pypi_0 pypi
image-classifiers 1.0.0 pypi_0 pypi
imageio 2.15.0 pypi_0 pypi
imagesize 1.4.1 pypi_0 pypi
imgaug 0.4.0 pypi_0 pypi
imgstore 0.2.9 pypi_0 pypi
importlib-metadata 4.12.0 pypi_0 pypi
importlib-resources 5.9.0 pypi_0 pypi
iniconfig 1.1.1 pypi_0 pypi
intel-openmp 2022.0.0 haa95532_3663
ipykernel 6.15.1 pypi_0 pypi
ipython 7.34.0 pypi_0 pypi
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 7.7.1 pypi_0 pypi
jedi 0.17.2 pypi_0 pypi
jinja2 3.1.2 pypi_0 pypi
joblib 1.1.0 pypi_0 pypi
jpeg 9e h2bbff1b_0
jsmin 3.0.1 pypi_0 pypi
json5 0.9.9 pypi_0 pypi
jsonpickle 1.2 pypi_0 pypi
jsonschema 4.9.1 pypi_0 pypi
jupyter-cache 0.4.3 pypi_0 pypi
jupyter-client 7.3.4 pypi_0 pypi
jupyter-core 4.11.1 pypi_0 pypi
jupyter-server 1.13.5 pypi_0 pypi
jupyter-server-mathjax 0.2.6 pypi_0 pypi
jupyter-sphinx 0.3.2 pypi_0 pypi
jupyterlab 3.3.4 pypi_0 pypi
jupyterlab-pygments 0.2.2 pypi_0 pypi
jupyterlab-server 2.15.0 pypi_0 pypi
jupyterlab-widgets 1.1.1 pypi_0 pypi
keras 2.8.0 pypi_0 pypi
keras-applications 1.0.8 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
keyring 23.7.0 pypi_0 pypi
kiwisolver 1.4.4 pypi_0 pypi
libblas 3.8.0 20_mkl conda-forge
libcblas 3.8.0 20_mkl conda-forge
libclang 14.0.6 pypi_0 pypi
libiconv 1.16 h2bbff1b_2
liblapack 3.8.0 20_mkl conda-forge
libpng 1.6.37 h2a8f88b_0
libtiff 4.2.0 he0120a3_1
libwebp 1.2.2 h2bbff1b_0
libxml2 2.9.14 h0ad7f3c_0
libxslt 1.1.35 h2bbff1b_0
libzlib 1.2.12 h8ffe710_2 conda-forge
linkify-it-py 2.0.0 pypi_0 pypi
livereload 2.6.3 pypi_0 pypi
lz4-c 1.9.3 h2bbff1b_1
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markdown 3.4.1 pypi_0 pypi
markdown-it-py 1.1.0 pypi_0 pypi
markupsafe 2.1.1 pypi_0 pypi
matplotlib 3.5.2 pypi_0 pypi
matplotlib-inline 0.1.3 pypi_0 pypi
mdit-py-plugins 0.2.8 pypi_0 pypi
mistune 0.8.4 pypi_0 pypi
mkl 2020.2 256
msys2-conda-epoch 20160418 1
mypy-extensions 0.4.3 pypi_0 pypi
myst-nb 0.13.2 pypi_0 pypi
myst-parser 0.15.2 pypi_0 pypi
nbclassic 0.4.3 pypi_0 pypi
nbclient 0.5.13 pypi_0 pypi
nbconvert 6.5.0 pypi_0 pypi
nbdime 3.1.1 pypi_0 pypi
nbformat 5.1.3 pypi_0 pypi
ndx-pose 0.1.1 pypi_0 pypi
nest-asyncio 1.5.5 pypi_0 pypi
networkx 2.6.3 pypi_0 pypi
nodeenv 1.7.0 pypi_0 pypi
notebook 6.4.12 pypi_0 pypi
notebook-shim 0.1.0 pypi_0 pypi
numpy 1.21.5 py37h2830a78_1 conda-forge
oauthlib 3.2.0 pypi_0 pypi
olefile 0.46 py37_0
opencv-python 4.6.0.66 pypi_0 pypi
opencv-python-headless 4.5.5.62 pypi_0 pypi
openssl 1.1.1q h2bbff1b_0
opt-einsum 3.3.0 pypi_0 pypi
packaging 21.3 pypi_0 pypi
pandas 1.3.5 py37h9386db6_0 conda-forge
pandocfilters 1.5.0 pypi_0 pypi
parso 0.7.1 pypi_0 pypi
pathspec 0.9.0 pypi_0 pypi
pickleshare 0.7.5 pypi_0 pypi
pillow 8.4.0 py37hd45dc43_0
pip 22.0.3 pyhd8ed1ab_0 conda-forge
pkginfo 1.8.3 pypi_0 pypi
pkgutil-resolve-name 1.3.10 pypi_0 pypi
platformdirs 2.5.2 pypi_0 pypi
pluggy 1.0.0 pypi_0 pypi
pre-commit 2.20.0 pypi_0 pypi
prometheus-client 0.14.1 pypi_0 pypi
prompt-toolkit 3.0.30 pypi_0 pypi
protobuf 3.19.4 pypi_0 pypi
psutil 5.9.1 pypi_0 pypi
py 1.11.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycparser 2.21 pypi_0 pypi
pydata-sphinx-theme 0.8.1 pypi_0 pypi
pygithub 1.55 pypi_0 pypi
pygments 2.12.0 pypi_0 pypi
pyjwt 2.4.0 pypi_0 pypi
pykalman 0.9.5 pypi_0 pypi
pynacl 1.5.0 pypi_0 pypi
pynwb 2.1.0 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
pyreadline 2.1 py37_1
pyrsistent 0.18.1 pypi_0 pypi
pyside2 5.14.1 pypi_0 pypi
pytest 7.1.2 pypi_0 pypi
pytest-cov 3.0.0 pypi_0 pypi
pytest-qt 4.1.0 pypi_0 pypi
pytest-xvfb 2.0.0 pypi_0 pypi
python 3.7.13 h6244533_0
python-dateutil 2.8.2 pyhd3eb1b0_0
python-rapidjson 1.8 pypi_0 pypi
python_abi 3.7 2_cp37m conda-forge
pytz 2022.1 py37haa95532_0
pytz-deprecation-shim 0.1.0.post0 pypi_0 pypi
pyvirtualdisplay 3.0 pypi_0 pypi
pywavelets 1.3.0 pypi_0 pypi
pywin32 304 pypi_0 pypi
pywin32-ctypes 0.2.0 pypi_0 pypi
pywinpty 1.1.6 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
pyzmq 23.2.0 pypi_0 pypi
qimage2ndarray 1.8.3 pypi_0 pypi
qt 5.12.9 h556501e_6 conda-forge
readme-renderer 35.0 pypi_0 pypi
regex 2022.7.25 pypi_0 pypi
requests 2.28.1 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
requests-toolbelt 0.9.1 pypi_0 pypi
rfc3986 2.0.0 pypi_0 pypi
rich 10.16.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
ruamel-yaml 0.17.21 pypi_0 pypi
ruamel-yaml-clib 0.2.6 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scikit-learn 1.0.2 pypi_0 pypi
scikit-video 1.1.11 pypi_0 pypi
scipy 1.7.3 py37hb6553fb_0 conda-forge
seaborn 0.11.2 pypi_0 pypi
segmentation-models 1.0.1 pypi_0 pypi
send2trash 1.8.0 pypi_0 pypi
setuptools 58.0.4 py37haa95532_0
shapely 1.7.1 py37h06580b3_0
shiboken2 5.14.1 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
sleap 1.2.6 dev_0 <develop>
smmap 5.0.0 pypi_0 pypi
sniffio 1.2.0 pypi_0 pypi
snowballstemmer 2.2.0 pypi_0 pypi
soupsieve 2.3.2.post1 pypi_0 pypi
sphinx 4.5.0 pypi_0 pypi
sphinx-autobuild 2021.3.14 pypi_0 pypi
sphinx-book-theme 0.3.3 pypi_0 pypi
sphinx-copybutton 0.5.0 pypi_0 pypi
sphinx-togglebutton 0.3.2 pypi_0 pypi
sphinxcontrib-applehelp 1.0.2 pypi_0 pypi
sphinxcontrib-devhelp 1.0.2 pypi_0 pypi
sphinxcontrib-htmlhelp 2.0.0 pypi_0 pypi
sphinxcontrib-jsmath 1.0.1 pypi_0 pypi
sphinxcontrib-qthelp 1.0.3 pypi_0 pypi
sphinxcontrib-serializinghtml 1.1.5 pypi_0 pypi
sqlalchemy 1.4.39 pypi_0 pypi
sqlite 3.39.0 h2bbff1b_0
tensorboard 2.8.0 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorflow 2.8.2 pypi_0 pypi
tensorflow-estimator 2.8.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.26.0 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
terminado 0.15.0 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tifffile 2021.11.2 pypi_0 pypi
tinycss2 1.1.1 pypi_0 pypi
tk 8.6.12 h2bbff1b_0
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
tornado 6.2 pypi_0 pypi
tqdm 4.64.0 pypi_0 pypi
traitlets 5.3.0 pypi_0 pypi
twine 3.3.0 pypi_0 pypi
typed-ast 1.5.4 pypi_0 pypi
typing-extensions 4.3.0 pypi_0 pypi
tzdata 2022.1 pypi_0 pypi
tzlocal 4.2 pypi_0 pypi
uc-micro-py 1.0.1 pypi_0 pypi
urllib3 1.26.11 pypi_0 pypi
vc 14.2 h21ff451_1
virtualenv 20.16.3 pypi_0 pypi
vs2015_runtime 14.27.29016 h5e58377_2
wcwidth 0.2.5 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.3.3 pypi_0 pypi
werkzeug 2.2.1 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0
widgetsnbextension 3.6.1 pypi_0 pypi
wincertstore 0.2 py37haa95532_2
wrapt 1.14.1 pypi_0 pypi
xz 5.2.5 h8cc25b3_1
zipp 3.8.1 pypi_0 pypi
zlib 1.2.12 h8cc25b3_2
zstd 1.5.2 h19a0ad4_0
Logs
Traceback (most recent call last):
File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\gui\app.py", line 1709, in _handle_model_overlay_command
predictor = DataOverlay.make_predictor(filename)
File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\gui\overlays\base.py", line 118, in make_predictor
return VisualPredictor.from_trained_models(filename)
File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\nn\inference.py", line 464, in from_trained_models
return cls(config=cfg, model=model)
TypeError: Can't instantiate abstract class VisualPredictor with abstract methods _initialize_inference_model, is_grayscale
Screenshots
How to reproduce
- Open GUI
- Click on Predict > Visualize Model Outputs...
- Select model
- See error
Discussed in https://github.com/talmolab/sleap/discussions/907
Originally posted by DEQ21 August 11, 2022 I recently have started the process of labeling and using SLEAP to cover most of the labels in the video. However, I am trying to generate the confidence graphs and velocity graphs that appear in the validation step. When I press visualize model outputs I get this : "TypeError: Can't instantiate abstract class VisualPredictor with abstract methods _initialize_inference_model, is_grayscale" or I get a heading error. Could someone let me know how I should proceed?
Problem Analysis
It seems we updated the Predictor base class with abstract methods while leaving VisualPredictor behind, namely is_grayscale and _inititalize_inference_model are the abstract methods that are not present in VisualPredictor (but should be). If we create dummy methods for VisualPredictor s.a.
def is_grayscale(self):
pass
def _inititalize_inference_model(self):
pass
then, VisualPredictor starts working again! It is very slow to switch between frames, and I don't see a way to exit the predictor..... In fact, there is a comment
# TODO: Rewrite this class.
which will likely be the focus of this PR.
Proposal
Rewrite the VisualPredictor class to include the neccessary abstract methods. Also add an exit key/method and speed-up functionality if possible (data cache so that we don't redo predictions).
Observed inference + postprocessing times
INFO:sleap.nn.inference:Finished 1 examples in 1.62 seconds (inference +
INFO:sleap.nn.inference:examples/s = 0.6168956864408537
INFO:sleap.nn.inference:Finished 1 examples in 1.59 seconds (inference +
INFO:sleap.nn.inference:examples/s = 0.6301173038943354
INFO:sleap.nn.inference:Finished 1 examples in 1.58 seconds (inference +
INFO:sleap.nn.inference:examples/s = 0.6349023548871147
ostprocessing)
INFO:sleap.nn.inference:examples/s = 0.6398051374733892
INFO:sleap.nn.inference:Finished 1 examples in 1.57 seconds (inference + postprocessing)
INFO:sleap.nn.inference:examples/s = 0.6361333659920815