Inconsistent results between different python environment.

[taroushirani]

Hello, I got inconsistent results from the same data when I changed the python environment.

test code:

import numpy as np
from fastdtw import fastdtw

x = [1, 6, 6, 33, 35]
y = [1, 6, 6, 33, 35]

d, path = fastdtw(x, y, radius=len(x))

print(f"d: {d}")
print(f"path: {path}")

environment a: WinPython 3.8.5 on Windows 10 Pro 64bit

$ python --version
Python 3.8.5

$ pip list | grep fastdtw
fastdtw                0.3.4

$ python test_fastdtw.py
d: 0.0
path: [(0, 0), (1, 1), (1, 2), (2, 2), (3, 3), (4, 4)]

environment b: Debian GNU/Linux(buster) (WSL)

$ python -V
Python 3.7.3

$ pip list | grep fastdtw
fastdtw                0.3.4

$ python test_fastdtw.py
d: 0.0
path: [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)]

The result of latter is what I need. Could anyone please advise me to avoid this inconsistency?

[taroushirani]

I found that pip of WinPython installed a pure python version of fastdtw. When I built fastdtw from source and changed to use the compiled version of fastdtw, the results of two environments matched. So this phenomenon results from the difference between two fastdtw implementation(pure python version or c++ version).

[slaypni]

Thank you for the information. Maybe better to change setup.py to install cython version always. If someone knows how to do that without dropping portability (architecture compatibility), I would like to know it.

[apacha]

I experience inconsistencies when using FastDTW too. I have two Mac Books with a very similar environment, but the results are different, causing some of my tests to fail:

My test code is simply

from fastdtw import fastdtw

x = [1, 4, 2, 6, 3, 3, 3, 5, 7, 1, 2, 1]
y = [1, 4, 6, 3, 3, 3, 3, 5, 8, 1, 2, 1, 9]
distance, path = fastdtw(x, y)
print(distance)
print(path)

Environment 1

(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % python --version
Python 3.8.2
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip --version
pip 21.0.1 from /Users/alex/Repositories/omr-reconstruction/venv/lib/python3.8/site-packages/pip (python 3.8)
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip freeze | grep fastdtw
fastdtw==0.3.4
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip freeze | grep numpy  
numpy==1.20.1

Result:

11.0
[(0, 0), (1, 1), (2, 1), (3, 2), (4, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (11, 12)]

Environment 2

(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % python --version
Python 3.8.7
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip --version
pip 21.0.1 from /Users/alex/Repositories/omr-reconstruction/venv/lib/python3.8/site-packages/pip (python 3.8)
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip freeze | grep fastdtw
fastdtw==0.3.4
(venv) (base) alex@Alexanders-MacBook-Pro omr-reconstruction % pip freeze | grep numpy
numpy==1.20.1

Result:

11.0
[(0, 0), (1, 1), (2, 1), (3, 2), (4, 3), (4, 4), (4, 5), (4, 6), (5, 6), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (11, 12)]

Do you have any idea what might be causing these inconsistencies? If more system information is needed, I will gladly provide it.

[tsh11na]

As written in https://github.com/slaypni/fastdtw/issues/46#issuecomment-692341110, that inconsistency is due to wheter the installed version internally uses Cython or not.

In #62, I modified version description for users to identify the actual installed version. If your version uses Cython, the description will be "0.3.4+Cython", otherwise "0.3.4+raw".

---

As far as I've investigated, the differences between Cython and Python versions were:

• The Cython version doesn't make redundant paths when maching two identical graphs.
• The Cython version is dramatically faster (10x in my environment) than the native Python version.
• The Cython version worked with Python 3.8 (3.8.10) but NOT with Python 3.10.

[mrodencal-tbiov]

@tsh11na can you please share your environment set up (which libs you have installed & what version they are at) to get the cython version to work on Python 3.8? I can't get the cython version it to build at all in a codespace venv

[tsh11na]

@mrodencal-tbiov Hello! I've confirmed that the Cython version works with Python 3.8.10 (venv environment). You may need to install Cython (pip install cython) before installing fastdtw and other packages.

The packages I've installed are:

Package             Version     
------------------- ------------
commonmark          0.9.1       
contourpy           1.0.7       
cycler              0.11.0      
Cython              0.29.33     
fastdtw             0.3.4+cython
fonttools           4.38.0      
importlib-resources 5.12.0      
kiwisolver          1.4.4       
markdown-it-py      2.2.0       
matplotlib          3.7.0       
mdurl               0.1.2       
numpy               1.24.2      
packaging           23.0        
pandas              1.5.3       
Pillow              9.4.0       
pip                 20.0.2      
pkg-resources       0.0.0       
Pygments            2.14.0      
pyparsing           3.0.9       
python-dateutil     2.8.2       
pytz                2022.7.1    
rich                13.3.1      
seaborn             0.12.2      
setuptools          44.0.0      
six                 1.16.0      
typing-extensions   4.5.0       
wheel               0.38.4      
wslPath             0.3.1       
zipp                3.15.0      

[mrodencal-tbiov]

@tsh11na I figured out my problem, and seeing how I stumbled onto it by accident I figured I will post here so hopefully future ppl using this library don't lose days to this like I have.

I was installing all of the requirements from a requirements.txt and using a make file to build the project. You were right that you need to have all of the libraries fastdtw uses installed BEFORE installing it, even if you use a requirements.txt and have fastdtw listed at the bottom of the list. This means that your Makefile needs to install numpy before trying to install fastdtw with pip.

steps I used to get repeatability:

1. uninstall all of the pip packages to start with a clean slate "pip freeze | xargs pip uninstall -y"
2. upgrade pip to the latest "pip install --upgrade pip"
3. install your requirements.txt which must include numpy "pip install --no-cache-dir -r requirements.txt"
4. install fastdtw "pip install fastdtw==0.3.4"

I also checked and you do not need Cython installed for it to build properly. and I was able to repeat this successfully in both python version 3.8.10 and python version 3.9.16

The Makefile commands to successfully install fastdtw for your ci-cd piplines are as follows: install: #install commands pip install --upgrade pip &&
pip install --no-cache-dir -r requirements.txt pip install fastdtw==0.3.4

Hope this helps someone in the future.