elephant icon indicating copy to clipboard operation
elephant copied to clipboard

[Bug] Tests failing on aarch64: `test_welch_psd_multidim_input`, `test_welch_cohere_multidim_input`

Open sanjayankur31 opened this issue 2 years ago • 5 comments

Describe the bug

When building 0.11.0 for Fedora, these two tests fail on aarch64

To Reproduce

  1. Buiild and run pytest on an aarch64 system

Expected behavior

Tests should pass

Environment

  • OS (e.g., Linux): Fedora Linux (rawhide, 37)
  • How you installed elephant (conda, pip, source): From source
  • Python version: 3.10
  • neo python package version: 0.10.2
  • elephant python package version: 0.11.0
  • (Any additional python package you want to include here)

Logs:

Output for convenience:

=================================== FAILURES ===================================
________________ WelchPSDTestCase.test_welch_psd_multidim_input ________________
self = <elephant.test.test_spectral.WelchPSDTestCase testMethod=test_welch_psd_multidim_input>
    def test_welch_psd_multidim_input(self):
        # generate multidimensional data
        num_channel = 4
        data_length = 5000
        sampling_period = 0.001
        noise = np.random.normal(size=(num_channel, data_length))
        data_np = np.array(noise)
        # Since row-column order in AnalogSignal is different from the
        # conventional one, `data_np` needs to be transposed when its used to
        # define an AnalogSignal
        data_neo = n.AnalogSignal(data_np.T,
                                  sampling_period=sampling_period * pq.s,
                                  units='mV')
        data_neo_1dim = n.AnalogSignal(data_np[0],
                                       sampling_period=sampling_period * pq.s,
                                       units='mV')
    
        # check if the results from different input types are identical
        freqs_np, psd_np = elephant.spectral.welch_psd(data_np,
                                                       fs=1 / sampling_period)
        freqs_neo, psd_neo = elephant.spectral.welch_psd(data_neo)
        freqs_neo_1dim, psd_neo_1dim = elephant.spectral.welch_psd(
            data_neo_1dim)
        self.assertTrue(np.all(freqs_np == freqs_neo))
        self.assertTrue(np.all(psd_np == psd_neo))
>       self.assertTrue(np.all(psd_neo_1dim == psd_neo[0]))
E       AssertionError: False is not true
elephant/test/test_spectral.py:159: AssertionError
_____________ WelchCohereTestCase.test_welch_cohere_multidim_input _____________
self = <elephant.test.test_spectral.WelchCohereTestCase testMethod=test_welch_cohere_multidim_input>
    def test_welch_cohere_multidim_input(self):
        # generate multidimensional data
        num_channel = 4
        data_length = 5000
        sampling_period = 0.001
        x_np = np.array(np.random.normal(size=(num_channel, data_length)))
        y_np = np.array(np.random.normal(size=(num_channel, data_length)))
        # Since row-column order in AnalogSignal is different from the
        # convention in NumPy/SciPy, `data_np` needs to be transposed when its
        # used to define an AnalogSignal
        x_neo = n.AnalogSignal(x_np.T, units='mV',
                               sampling_period=sampling_period * pq.s)
        y_neo = n.AnalogSignal(y_np.T, units='mV',
                               sampling_period=sampling_period * pq.s)
        x_neo_1dim = n.AnalogSignal(x_np[0], units='mV',
                                    sampling_period=sampling_period * pq.s)
        y_neo_1dim = n.AnalogSignal(y_np[0], units='mV',
                                    sampling_period=sampling_period * pq.s)
    
        # check if the results from different input types are identical
        freqs_np, coherency_np, phase_lag_np = elephant.spectral \
            .welch_coherence(x_np, y_np, fs=1 / sampling_period)
        freqs_neo, coherency_neo, phase_lag_neo = \
            elephant.spectral.welch_coherence(x_neo, y_neo)
        freqs_neo_1dim, coherency_neo_1dim, phase_lag_neo_1dim = \
            elephant.spectral.welch_coherence(x_neo_1dim, y_neo_1dim)
        self.assertTrue(np.all(freqs_np == freqs_neo))
        self.assertTrue(np.all(coherency_np.T == coherency_neo))
        self.assertTrue(np.all(phase_lag_np.T == phase_lag_neo))
>       self.assertTrue(
            np.all(coherency_neo_1dim[:, 0] == coherency_neo[:, 0]))
E       AssertionError: False is not true
elephant/test/test_spectral.py:468: AssertionError

The tests passed on all other arches we support: x86_64, i686, s390x, ppc64le. Here's the temporary build (will be garbage collected in a few days):

https://koji.fedoraproject.org/koji/taskinfo?taskID=85975318

sanjayankur31 avatar Apr 20 '22 16:04 sanjayankur31

Hey, thanks for reporting this @sanjayankur31.

We will look into this in connection with issue #410 , since both issues are related to an execution on a different architecture. So far, neither architecture has been the focus of our attention during elephant development, but we consider taking this into account for the future.

Moritz-Alexander-Kern avatar Apr 21 '22 14:04 Moritz-Alexander-Kern

Thanks very much. Please do let me know if I can help in any way. I guess it's also OK for elephant to not support these arches if the dev team does not want to target them, just as long as this is noted somewhere so that users are aware.

sanjayankur31 avatar Apr 21 '22 16:04 sanjayankur31

  • add note to docs: elephant currently only has active support for the x86_64 architecture.
  • more recent build of elephant succeeded, see: https://koji.fedoraproject.org/koji/taskinfo?taskID=91845461
  • this might be related to scipy

Feel free to reopen this, if it comes up again.

Moritz-Alexander-Kern avatar Dec 05 '22 11:12 Moritz-Alexander-Kern

Another on aarch64 with 0.12.0, noting here for records:

______ MultitaperCoherenceTestCase.test_multitaper_cohere_perfect_cohere _______
self = <elephant.test.test_spectral.MultitaperCoherenceTestCase testMethod=test_multitaper_cohere_perfect_cohere>
    def test_multitaper_cohere_perfect_cohere(self):
        # Generate dummy data
        data_length = 10000
        sampling_period = 0.001
        signal_freq = 100.0
        noise = np.random.normal(size=(1, data_length))
        time_points = np.arange(0, data_length * sampling_period,
                                sampling_period)
        signal = np.cos(2 * np.pi * signal_freq * time_points) + noise
    
        # Estimate coherence and phase lag with the multitaper method
        freq1, coh, phase_lag = elephant.spectral.multitaper_coherence(
            signal,
            signal,
            fs=1/sampling_period,
            n_segments=16)
    
        self.assertTrue((coh == np.ones(coh.size)).all())
>       self.assertTrue((phase_lag == np.zeros(phase_lag.size)).all())
E       AssertionError: False is not true
elephant/test/test_spectral.py:867: AssertionError

sanjayankur31 avatar Feb 27 '23 14:02 sanjayankur31

Thank you for reporting this. Indeed, numerical precision is an issue for these unit tests. We'll try to come up with a better solution for handling these cases.

mdenker avatar Mar 02 '23 08:03 mdenker

Addressed in #573 .

Feel free to reopen at any time.

Moritz-Alexander-Kern avatar Apr 04 '24 12:04 Moritz-Alexander-Kern