mojo icon indicating copy to clipboard operation
mojo copied to clipboard

[BUG] Benchmarking multiple methods throws thread lock object error.

Open lucapantea opened this issue 1 year ago • 1 comments

Bug Description

When benchmarking different implementations of matrix multiplication as shown in the snippet below (taken from Matmul.ipynb), I get an error related to the use of a thread lock object and its context manager functionality

Steps to Reproduce

  1. First, declare the methods to be benchmarked.
%%python
import numpy as np

def matmul_python(C, A, B):
    for m in range(C.rows):
        for n in range(C.cols):
            for k in range(A.cols):
                C[m, n] += A[m, k] * B[k, n]
                
def matmul_python_optimized(C, A, B):
    C += A @ B
    
def matmul_python_numpy(C, A, B):
    C += np.matmul(A, B)
  1. Create respective benchmarking methods for each function
%%python
import numpy as np
from timeit import timeit

class Matrix:
    ...  # same as in the Matmul.ipynb notebook

def benchmark_matmul_python(M, N, K):
    ...  # same as in the Matmul.ipynb notebook
    
def benchmark_matmul_python_optimized(M, N, K):
    A_np = np.random.rand(M, K)
    B_np = np.random.rand(K, N)
    C_np = np.zeros((M, N))
    secs_python = timeit(lambda: matmul_python_optimized(C_np, A_np, B_np), number=2)/2
    gflops_python = ((2*M*N*K)/secs_python) / 1e9
    print("PYTHON optimized:", gflops_python, "GFLOP/s")

def benchmark_matmul_python_numpy(M, N, K):
    A_np = np.random.rand(M, K)
    B_np = np.random.rand(K, N)
    C_np = np.zeros((M, N))
    secs_np = timeit(lambda: matmul_python_numpy(C_np, A_np, B_np), number=2)/2
    gflops_np = ((2*M*N*K)/secs_np) / 1e9
    print("NUMPY optimized:", gflops_np, "GFLOP/s")
    return gflops
  1. Create a cell to benchmark each solution
python_gflops = benchmark_matmul_python(128, 128, 128).to_f64()
python_gflops_optimized = benchmark_matmul_python_optimized(128, 128, 128).to_f64()
python_gflops_numpy = benchmark_matmul_python_numpy(128, 128, 128).to_f64()
  1. After running the first benchmark, the second function call (to benchmark_matmul_python_optimized) throws the following error:
Error: <built-in method __enter__ of _thread.lock object at 0x7f9075d5ccf0> returned a result with an error set

Context

11:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 10:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 9:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 8:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 7:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 6:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 5:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 4:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 3:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 2:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cd3c8_111b_485d_8f75_6a5a1b30f202.slice/cri-containerd-1a087d8e5459ac60fe2739da93023ade5253d5863b611b9c7f6b993e4a2cb797.scope 0::/

lucapantea avatar May 18 '23 19:05 lucapantea

This seems odd, and I'm not sure if it's due to the notebook environment or something is being mishandled on the Python interop side. FYI @stumpOS @walter-erquinigo @River707

Mogball avatar May 22 '23 16:05 Mogball

We've made many changes to the REPL in the past months, and this shouldn't happen anymore. Please reopen if the issue persists.

walter-erquinigo avatar Mar 28 '24 20:03 walter-erquinigo