[WIP] 21.10 Notebook Testing Report

[Nicholas-7]

Describe the bug Discovered errors in a few notebooks relating to the cuSignal library after running them in the 21.10 stable release. The errors were tested and found in both CentOS and Ubuntu operating systems. Not sure if these errors are a result of possible updates to the codebase or if it was an uncaught bug.

Steps/Code to reproduce bug Steps to reproduce the behavior: 1.Go to RAPIDS Sample Notebooks and clone the 21.10 branch 2.Click on the cuSignal folder 3.Run all the cells of the notebooks to produce the examples illustrated below

Expected behavior There will be several examples that will create an error. Many examples miss details that could aide in implementation. The code may be a few commits behind from the 21.10 repo.

Environment details (please complete the following information):

• Environment location: Docker
• Linux Distro/Architecture: Ubuntu 20.04 amd64 and CentOS 8
• GPU Model/Driver: GV100, 450.142.00
• CUDA: 11.0
• Method of Library install: Docker Install

Additional context Examples of Discrepancies:

Example # 1 sdr_integration

try:
        import RtlSdr; print('RtlSdr Version:', RtlSdr.__version__)  
except ModuleNotFoundError:
        !conda install -c conda-forge rtl-sdr -y
        import RtlSdr; print('RtlSdr Version:', RtlSdr.__version__)

# CPU Powered Libraries
import numpy as np
from scipy import signal

# GPU Powered Libraries
import cupy as cp
import cusignal

# RTL-SDR Support
#from rtlsdr import RtlSdr

# Plotting
import matplotlib.pyplot as plt

Where after discovering the above I tried to remedy by reinstalling RtlSdr using the uncommented code.

Error thrown below:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-ed9e90400c9c> in <module>
      1 try:
----> 2         import RtlSdr; print('RtlSdr Version:', RtlSdr.__version__)
      3 except ModuleNotFoundError:

ModuleNotFoundError: No module named 'RtlSdr'

During handling of the above exception, another exception occurred:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-ed9e90400c9c> in <module>
      3 except ModuleNotFoundError:
      4         get_ipython().system('conda install -c conda-forge rtl-sdr -y')
----> 5         import RtlSdr; print('RtlSdr Version:', RtlSdr.__version__)
      6 
      7 # CPU Powered Libraries

ModuleNotFoundError: No module named 'RtlSdr'

Example # 2 sdr_wfm_demod

try:
        import SoapySDR; print('SoapySDR Version:', SoapySDR.__version__)  
except ModuleNotFoundError:
        !conda install -c conda-forge soapysdr -y
        import SoapySDR; print('SoapySDR Version:', SoapySDR.__version__)

from SoapySDR import *
#import SoapySDR
import pyaudio
import signal
import queue
import cupy as cp
import cusignal as sig

Where after discovering the above I tried to remedy by reinstalling SoapySDR using the uncommented code.

Error thrown below:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-043677171a3a> in <module>
----> 1 from SoapySDR import *
      2 import SoapySDR
      3 import pyaudio
      4 import signal
      5 import queue

ModuleNotFoundError: No module named 'SoapySDR'

Example # 3 convolution_examples

%%timeit
ggrad = cusignal.convolve2d(gsig, gfilt, use_numba=True)

Where use_numba=True isnt recognized as a parameter for convolve2d()

Error thrown below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-17-cf87f78265d3> in <module>
----> 1 get_ipython().run_cell_magic('timeit', '', 'ggrad = cusignal.convolve2d(gsig, gfilt, use_numba=True)\n')

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2360             with self.builtin_trap:
   2361                 args = (magic_arg_s, cell)
-> 2362                 result = fn(*args, **kwargs)
   2363             return result
   2364 

<decorator-gen-53> in timeit(self, line, cell, local_ns)

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magics/execution.py in timeit(self, line, cell, local_ns)
   1158             for index in range(0, 10):
   1159                 number = 10 ** index
-> 1160                 time_number = timer.timeit(number)
   1161                 if time_number >= 0.2:
   1162                     break

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magics/execution.py in timeit(self, number)
    167         gc.disable()
    168         try:
--> 169             timing = self.inner(it, self.timer)
    170         finally:
    171             if gcold:

<magic-timeit> in inner(_it, _timer)

TypeError: convolve2d() got an unexpected keyword argument 'use_numba'

A second instance of the similar issue filtering_examples

%%timeit
cf = cusignal.resample_poly(gy, resample_up, resample_down, window=('kaiser', 0.5), use_numba=True)

Where use_numba=True isnt recognized as a parameter for resample_poly()

Error thrown below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-ae7e2bba340d> in <module>
----> 1 get_ipython().run_cell_magic('timeit', '', "cf = cusignal.resample_poly(gy, resample_up, resample_down, window=('kaiser', 0.5), use_numba=True)\n")

/opt/conda/envs/rapids/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2360             with self.builtin_trap:
   2361                 args = (magic_arg_s, cell)
-> 2362                 result = fn(*args, **kwargs)
   2363             return result
   2364 

<decorator-gen-53> in timeit(self, line, cell, local_ns)

/opt/conda/envs/rapids/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

/opt/conda/envs/rapids/lib/python3.7/site-packages/IPython/core/magics/execution.py in timeit(self, line, cell, local_ns)
   1158             for index in range(0, 10):
   1159                 number = 10 ** index
-> 1160                 time_number = timer.timeit(number)
   1161                 if time_number >= 0.2:
   1162                     break

/opt/conda/envs/rapids/lib/python3.7/site-packages/IPython/core/magics/execution.py in timeit(self, number)
    167         gc.disable()
    168         try:
--> 169             timing = self.inner(it, self.timer)
    170         finally:
    171             if gcold:

<magic-timeit> in inner(_it, _timer)

TypeError: resample_poly() got an unexpected keyword argument 'use_numba'

Example # 4 io_examples

with open(meta_file, 'r') as f:
    md = json.loads(f.read())

if md['_metadata']['global']['core:datatype'] == 'cf32':
    data_type = np.complex64

Where /data/oracle/KRI-16Devices-RawData/2ft/WiFi_air_X310_3123D7B_2ft_run1.sigmf-meta is not available

Error thrown below:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-3-813c154d9aea> in <module>
----> 1 with open(meta_file, 'r') as f:
      2     md = json.loads(f.read())
      3 
      4 if md['_metadata']['global']['core:datatype'] == 'cf32':
      5     data_type = np.complex64

FileNotFoundError: [Errno 2] No such file or directory: '/data/oracle/KRI-16Devices-RawData/2ft/WiFi_air_X310_3123D7B_2ft_run1.sigmf-meta'

Example # 5 wavelets_examples

%%timeit
gcwt = cusignal.cwt(gsig, cusignal.ricker, cp.arange(1,31))

Where we seem at one time to be expecting an integer.

Error thrown below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-85178b60d8e0> in <module>
----> 1 get_ipython().run_cell_magic('timeit', '', 'gcwt = cusignal.cwt(gsig, cusignal.ricker, cp.arange(1,31))\n')

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2360             with self.builtin_trap:
   2361                 args = (magic_arg_s, cell)
-> 2362                 result = fn(*args, **kwargs)
   2363             return result
   2364 

<decorator-gen-53> in timeit(self, line, cell, local_ns)

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magics/execution.py in timeit(self, line, cell, local_ns)
   1158             for index in range(0, 10):
   1159                 number = 10 ** index
-> 1160                 time_number = timer.timeit(number)
   1161                 if time_number >= 0.2:
   1162                     break

/opt/conda/envs/rapids/lib/python3.8/site-packages/IPython/core/magics/execution.py in timeit(self, number)
    167         gc.disable()
    168         try:
--> 169             timing = self.inner(it, self.timer)
    170         finally:
    171             if gcold:

<magic-timeit> in inner(_it, _timer)

/opt/conda/envs/rapids/lib/python3.8/site-packages/cusignal-21.10.0-py3.8.egg/cusignal/wavelets/wavelets.py in cwt(data, wavelet, widths)
    308     output = cp.empty([len(widths), len(data)])
    309     for ind, width in enumerate(widths):
--> 310         wavelet_data = wavelet(min(10 * width, len(data)), width)
    311         output[ind, :] = convolve(data, wavelet_data, mode="same")
    312     return output

/opt/conda/envs/rapids/lib/python3.8/site-packages/cusignal-21.10.0-py3.8.egg/cusignal/wavelets/wavelets.py in ricker(points, a)
    173 
    174     """
--> 175     return _ricker_kernel(a, size=points)
    176 
    177 

cupy/_core/_kernel.pyx in cupy._core._kernel.ElementwiseKernel.__call__()

TypeError: 'cupy._core.core.ndarray' object cannot be interpreted as an integer

Desired outcome cuSignal notebooks should be immediately ready to be replicated and implemented with less effort. Notebooks should be updated to reflect the commits made to the repositories during each release cycle. cuSignal functions and models work as expected.

Request impacts Our cuSignal notebooks are public and require accurate information - Medium Priority

@taureandyernv @fondaing @awthomp for awareness

[awthomp]

Thanks for the issue, @Nicholas-7.

I have no plans to add support to external dependencies to cusignal. This includes the packages rtlsdr, soapysdr, and the .sigmf file used to test I/O. These notebooks are meant to be a learning tool for the developer on how to get started with using cusignal "in the wild".

I'm okay if we want to disable these notebooks from running CI/CD or other creative solutions.

Thanks for bringing up the issue with use_numba. I've filed a PR to fix that.

Finally, for the cwt error, we recently filed a fix that should make it into the next release of cusignal that addresses this error.

[github-actions[bot]]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[taureandyernv]

@awthomp may i propose a possible resolution?

1. i'll add conda install -y -c conda-forge soapysdr rtl-sdr to the affected notebooks and PR it.
2. As for the sigm file, is that something we can make available via an S3 bucket? if so, i'll ask ops to add it and link to it, once I get your go-ahead.

That should close this up nicely without interfering too much. the issue we're trying to fix is not a CI/CD issue as much as a "I love cusignal and can't run these notebooks to see its capability in these areas, so I can imagine others" issue. SDR is something I want to get into soon and am in process of getting the certs :)

[github-actions[bot]]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[github-actions[bot]]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.