cugraph
cugraph copied to clipboard
Published
20 hours ago •
rapidsai
rapidsai
[BUG]: MG `ego_graph` Different from SG
Version
24.04
Which installation method(s) does this occur on?
Source
Describe the bug.
Currently, the MG implementation of ego_graph returns a value that differs from the SG implementation when passed multiple n values, aka seeds.
Minimum reproducible example
import cugraph
import cugraph.dask as dcg
from cugraph.datasets import karate
from cugraph.testing.mg_utils import (
start_dask_client, stop_dask_client
)
import pytest
from cugraph.testing import utils
from cudf.testing.testing import assert_frame_equal
if __name__ == "__main__":
# start dask
client, cluster = start_dask_client()
# Parameters to re-create failure
data = karate
directed = True
seeds = [0, 2]
radius = 1
# prep
data_path = data.get_path()
###################################################################
# SG
g = data.get_graph(create_using=cugraph.Graph(directed=directed))
###################################################################
# MG
blocksize = dcg.get_chunksize(data_path)
ddf = data.get_dask_edgelist()
dg = cugraph.Graph(directed=directed)
dg.from_dask_cudf_edgelist(
ddf,
source="src",
destination="dst",
edge_attr="wgt",
renumber=True,
store_transposed=True,
)
###################################################################
sg_cugraph_ego_graphs = cugraph.batched_ego_graphs(g, seeds=seeds, radius=radius)
mg_cugraph_ego_graphs = dcg.ego_graph(dg, seeds, radius)
sg_df, sg_offsets = sg_cugraph_ego_graphs
mg_df, mg_offsets = mg_cugraph_ego_graphs
mg_df = mg_df.compute()
# This Fails
assert_frame_equals(mg_df, sg_df)
# stop dask
stop_dask_client(client, cluster)
Relevant log output
obj = 'ColumnBase', message = 'values are different (100.0 %)'
left = {'[0, 118, 260]'}, right = {'[0, 142, 260]'}, diff = None
Environment details
<details><summary>Click here to see environment details</summary><pre>
**git***
commit 6c73718395a9499e4c01f93b7ab515ad191882c2 (HEAD -> b2404-fix-test-egonet-mg)
Author: Ralph Liu <[email protected]>
Date: Thu Feb 22 09:45:38 2024 -0800
Use Series.iloc to remove FutureWarning
**git submodules***
***OS Information***
DGX_NAME="DGX Server"
DGX_PRETTY_NAME="NVIDIA DGX Server"
DGX_SWBUILD_DATE="2023-03-27-13-31-04"
DGX_SWBUILD_VERSION="5.5.0"
DGX_COMMIT_ID="b2e06e0"
DGX_PLATFORM="DGX Server for DGX-1"
DGX_SERIAL_NUMBER="QTFCOU8310012"
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Linux dgx16 5.4.0-149-generic #166-Ubuntu SMP Tue Apr 18 16:51:45 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
***GPU Information***
Fri Feb 23 14:13:20 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100-SXM2... On | 00000000:06:00.0 Off | 0 |
| N/A 34C P0 57W / 300W | 15097MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 Tesla V100-SXM2... On | 00000000:07:00.0 Off | 0 |
| N/A 33C P0 43W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 Tesla V100-SXM2... On | 00000000:0A:00.0 Off | 0 |
| N/A 30C P0 42W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 Tesla V100-SXM2... On | 00000000:0B:00.0 Off | 0 |
| N/A 31C P0 43W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 Tesla V100-SXM2... On | 00000000:85:00.0 Off | 0 |
| N/A 32C P0 43W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 5 Tesla V100-SXM2... On | 00000000:86:00.0 Off | 0 |
| N/A 31C P0 42W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 6 Tesla V100-SXM2... On | 00000000:89:00.0 Off | 0 |
| N/A 32C P0 43W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 7 Tesla V100-SXM2... On | 00000000:8A:00.0 Off | 0 |
| N/A 32C P0 42W / 300W | 3MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2854958 C /opt/conda/bin/python3.10 752MiB |
| 0 N/A N/A 3658368 C /opt/conda/bin/python3.10 3338MiB |
| 0 N/A N/A 3839353 C ...conda3/envs/v5/bin/python 10986MiB |
+-----------------------------------------------------------------------------+
***CPU***
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 80
On-line CPU(s) list: 0-79
Thread(s) per core: 2
Core(s) per socket: 20
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
Stepping: 1
CPU MHz: 2776.545
CPU max MHz: 3600.0000
CPU min MHz: 1200.0000
BogoMIPS: 4390.25
Virtualization: VT-x
L1d cache: 1.3 MiB
L1i cache: 1.3 MiB
L2 cache: 10 MiB
L3 cache: 100 MiB
NUMA node0 CPU(s): 0-19,40-59
NUMA node1 CPU(s): 20-39,60-79
Vulnerability Itlb multihit: KVM: Mitigation: Split huge pages
Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown: Mitigation; PTI
Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d
***CMake***
/home/nfs/ralphl/miniforge3/envs/test-24.04/bin/cmake
cmake version 3.28.3
CMake suite maintained and supported by Kitware (kitware.com/cmake).
***g++***
/home/nfs/ralphl/miniforge3/envs/test-24.04/bin/g++
g++ (conda-forge gcc 11.4.0-5) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
***nvcc***
/home/nfs/ralphl/miniforge3/envs/test-24.04/bin/nvcc
/home/nfs/ralphl/miniforge3/envs/test-24.04/bin/nvcc: line 9: /bin/nvcc: No such file or directory
***Python***
/home/nfs/ralphl/miniforge3/envs/test-24.04/bin/python
Python 3.10.13
***Environment Variables***
PATH : /home/nfs/ralphl/miniforge3/envs/test-24.04/bin:/home/nfs/ralphl/miniforge3/condabin:/home/nfs/ralphl/.vscode-server/cli/servers/Stable-903b1e9d8990623e3d7da1df3d33db3e42d80eda/server/bin/remote-cli:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
LD_LIBRARY_PATH :
NUMBAPRO_NVVM :
NUMBAPRO_LIBDEVICE :
CONDA_PREFIX : /home/nfs/ralphl/miniforge3/envs/test-24.04
PYTHON_PATH :
***conda packages***
/home/nfs/ralphl/miniforge3/condabin/conda
# packages in environment at /home/nfs/ralphl/miniforge3/envs/test-24.04:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
_sysroot_linux-64_curr_repodata_hack 3 h69a702a_13 conda-forge
accessible-pygments 0.0.4 pyhd8ed1ab_0 conda-forge
aiohttp 3.9.3 py310h2372a71_0 conda-forge
aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
alabaster 0.7.16 pyhd8ed1ab_0 conda-forge
anyio 4.2.0 pyhd8ed1ab_0 conda-forge
argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py310h2372a71_4 conda-forge
arrow 1.3.0 pyhd8ed1ab_0 conda-forge
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
async-lru 2.0.4 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.3 pyhd8ed1ab_0 conda-forge
atk-1.0 2.38.0 hd4edc92_1 conda-forge
attrs 23.2.0 pyh71513ae_0 conda-forge
aws-c-auth 0.7.15 h70caa3e_0 conda-forge
aws-c-cal 0.6.9 h14ec70c_3 conda-forge
aws-c-common 0.9.12 hd590300_0 conda-forge
aws-c-compression 0.2.17 h572eabf_8 conda-forge
aws-c-event-stream 0.4.1 h17cd1f3_5 conda-forge
aws-c-http 0.8.0 hc6da83f_5 conda-forge
aws-c-io 0.14.3 h3c8c088_1 conda-forge
aws-c-mqtt 0.10.1 h0ef3971_3 conda-forge
aws-c-s3 0.5.0 h1b46bed_2 conda-forge
aws-c-sdkutils 0.1.14 h572eabf_0 conda-forge
aws-checksums 0.1.17 h572eabf_7 conda-forge
aws-crt-cpp 0.26.1 h33f84b2_9 conda-forge
aws-sdk-cpp 1.11.242 h65f022c_0 conda-forge
babel 2.14.0 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.3 pyha770c72_0 conda-forge
binutils 2.40 hdd6e379_0 conda-forge
binutils_impl_linux-64 2.40 hf600244_0 conda-forge
binutils_linux-64 2.40 hbdbef99_2 conda-forge
bleach 6.1.0 pyhd8ed1ab_0 conda-forge
bokeh 3.3.4 pyhd8ed1ab_0 conda-forge
breathe 4.35.0 pyhd8ed1ab_1 conda-forge
brotli 1.1.0 hd590300_1 conda-forge
brotli-bin 1.1.0 hd590300_1 conda-forge
brotli-python 1.1.0 py310hc6cd4ac_1 conda-forge
bzip2 1.0.8 hd590300_5 conda-forge
c-ares 1.26.0 hd590300_0 conda-forge
c-compiler 1.5.2 h0b41bf4_0 conda-forge
ca-certificates 2024.2.2 hbcca054_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.3.2 pyhd8ed1ab_0 conda-forge
cairo 1.18.0 h3faef2a_0 conda-forge
certifi 2024.2.2 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py310h2fee648_0 conda-forge
cfgv 3.3.1 pyhd8ed1ab_0 conda-forge
charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge
click 8.1.7 unix_pyh707e725_0 conda-forge
cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge
cmake 3.28.3 hcfe8598_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.2.1 pyhd8ed1ab_0 conda-forge
commonmark 0.9.1 py_0 conda-forge
contourpy 1.2.0 py310hd41b1e2_0 conda-forge
coverage 7.4.1 py310h2372a71_0 conda-forge
cubinlinker 0.3.0 py310hfdf336d_0 rapidsai
cuda-nvtx 12.3.101 0 nvidia
cuda-profiler-api 11.8.86 0 nvidia
cuda-python 11.8.3 py310h70a93da_0 conda-forge
cuda-version 11.8 h70ddcb2_2 conda-forge
cudatoolkit 11.8.0 h4ba93d1_13 conda-forge
cudf 24.04.00a406 cuda11_py310_240215_g99ed8b9977_406 rapidsai-nightly
cugraph 24.4.0 pypi_0 pypi
cupy 13.0.0 py310h189a05f_3 conda-forge
cupy-core 13.0.0 py310h506062a_3 conda-forge
cxx-compiler 1.5.2 hf52228f_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
cython 3.0.8 py310hc6cd4ac_0 conda-forge
cytoolz 0.12.3 py310h2372a71_0 conda-forge
dask 2024.1.1 pyhd8ed1ab_0 conda-forge
dask-core 2024.1.1 pyhd8ed1ab_0 conda-forge
dask-cuda 24.04.00a6 py310_240215_g21376a0_6 rapidsai-nightly
dask-cudf 24.04.00a406 cuda11_py310_240215_g99ed8b9977_406 rapidsai-nightly
debugpy 1.8.1 py310hc6cd4ac_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
distlib 0.3.8 pyhd8ed1ab_0 conda-forge
distributed 2024.1.1 pyhd8ed1ab_0 conda-forge
dlpack 0.5 h9c3ff4c_0 conda-forge
docutils 0.19 py310hff52083_1 conda-forge
doxygen 1.10.0 h661eb56_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge
execnet 2.0.2 pyhd8ed1ab_0 conda-forge
executing 2.0.1 pyhd8ed1ab_0 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
fastrlock 0.8.2 py310hc6cd4ac_2 conda-forge
filelock 3.13.1 pyhd8ed1ab_0 conda-forge
fmt 10.2.1 h00ab1b0_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_1 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.48.1 py310h2372a71_0 conda-forge
fqdn 1.5.1 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
frozenlist 1.4.1 py310h2372a71_0 conda-forge
fsspec 2024.2.0 pyhca7485f_0 conda-forge
future 0.18.3 pyhd8ed1ab_0 conda-forge
gcc 11.4.0 h7baecda_2 conda-forge
gcc_impl_linux-64 11.4.0 h7aa1c59_5 conda-forge
gcc_linux-64 11.4.0 hfd045f2_2 conda-forge
gdk-pixbuf 2.42.10 h829c605_4 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gflags 2.2.2 he1b5a44_1004 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
glog 0.6.0 h6f12383_0 conda-forge
gmock 1.14.0 ha770c72_1 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
graphviz 9.0.0 h78e8752_1 conda-forge
gtest 1.14.0 h00ab1b0_1 conda-forge
gtk2 2.24.33 h7f000aa_3 conda-forge
gts 0.7.6 h977cf35_4 conda-forge
gxx 11.4.0 h7baecda_2 conda-forge
gxx_impl_linux-64 11.4.0 h7aa1c59_5 conda-forge
gxx_linux-64 11.4.0 hfc1ae95_2 conda-forge
h11 0.14.0 pyhd8ed1ab_0 conda-forge
h2 4.1.0 pyhd8ed1ab_0 conda-forge
harfbuzz 8.3.0 h3d44ed6_0 conda-forge
hpack 4.0.0 pyh9f0ad1d_0 conda-forge
httpcore 1.0.3 pyhd8ed1ab_0 conda-forge
httpx 0.26.0 pyhd8ed1ab_0 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
icu 73.2 h59595ed_0 conda-forge
identify 2.5.34 pyhd8ed1ab_0 conda-forge
idna 3.6 pyhd8ed1ab_0 conda-forge
imagesize 1.4.1 pyhd8ed1ab_0 conda-forge
importlib-metadata 7.0.1 pyha770c72_0 conda-forge
importlib-resources 6.1.1 pyhd8ed1ab_0 conda-forge
importlib_metadata 7.0.1 hd8ed1ab_0 conda-forge
importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge
iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
ipykernel 6.29.2 pyhd33586a_0 conda-forge
ipython 8.21.0 pyh707e725_0 conda-forge
isoduration 20.11.0 pyhd8ed1ab_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.3 pyhd8ed1ab_0 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
json5 0.9.14 pyhd8ed1ab_0 conda-forge
jsonpointer 2.4 py310hff52083_3 conda-forge
jsonschema 4.21.1 pyhd8ed1ab_0 conda-forge
jsonschema-specifications 2023.12.1 pyhd8ed1ab_0 conda-forge
jsonschema-with-format-nongpl 4.21.1 pyhd8ed1ab_0 conda-forge
jupyter-lsp 2.2.2 pyhd8ed1ab_0 conda-forge
jupyter_client 8.6.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.7.1 py310hff52083_0 conda-forge
jupyter_events 0.9.0 pyhd8ed1ab_0 conda-forge
jupyter_server 2.12.5 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.5.2 pyhd8ed1ab_0 conda-forge
jupyterlab 4.1.1 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.3.0 pyhd8ed1ab_1 conda-forge
jupyterlab_server 2.25.2 pyhd8ed1ab_0 conda-forge
kaleido 0.2.1 pypi_0 pypi
kernel-headers_linux-64 3.10.0 h4a8ded7_13 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge
krb5 1.21.2 h659d440_0 conda-forge
lcms2 2.16 hb7c19ff_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20230802.1 cxx17_h59595ed_0 conda-forge
libarrow 14.0.2 he2c5238_5_cpu conda-forge
libarrow-acero 14.0.2 h59595ed_5_cpu conda-forge
libarrow-dataset 14.0.2 h59595ed_5_cpu conda-forge
libarrow-flight 14.0.2 hdc44a87_5_cpu conda-forge
libarrow-flight-sql 14.0.2 hfbc7f12_5_cpu conda-forge
libarrow-gandiva 14.0.2 hacb8726_5_cpu conda-forge
libarrow-substrait 14.0.2 hfbc7f12_5_cpu conda-forge
libblas 3.9.0 21_linux64_openblas conda-forge
libbrotlicommon 1.1.0 hd590300_1 conda-forge
libbrotlidec 1.1.0 hd590300_1 conda-forge
libbrotlienc 1.1.0 hd590300_1 conda-forge
libcblas 3.9.0 21_linux64_openblas conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcublas 11.11.3.6 0 nvidia
libcublas-dev 11.11.3.6 0 nvidia
libcudf 24.04.00a406 cuda11_240215_g99ed8b9977_406 rapidsai-nightly
libcufile 1.4.0.31 0 nvidia
libcufile-dev 1.4.0.31 0 nvidia
libcugraph 24.04.00a43 cuda11_240212_g38a8cdbb3_43 rapidsai-nightly
libcugraphops 24.04.00a12 cuda11_240214_ge65dfa63_12 rapidsai-nightly
libcurand 10.3.0.86 0 nvidia
libcurand-dev 10.3.0.86 0 nvidia
libcurl 8.5.0 hca28451_0 conda-forge
libcusolver 11.4.1.48 0 nvidia
libcusolver-dev 11.4.1.48 0 nvidia
libcusparse 11.7.5.86 0 nvidia
libcusparse-dev 11.7.5.86 0 nvidia
libdeflate 1.19 hd590300_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 hd590300_2 conda-forge
libevent 2.1.12 hf998b51_1 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-devel_linux-64 11.4.0 h922705a_105 conda-forge
libgcc-ng 13.2.0 h807b86a_5 conda-forge
libgd 2.3.3 h119a65a_9 conda-forge
libgfortran-ng 13.2.0 h69a702a_5 conda-forge
libgfortran5 13.2.0 ha4646dd_5 conda-forge
libglib 2.78.3 h783c2da_0 conda-forge
libgomp 13.2.0 h807b86a_5 conda-forge
libgoogle-cloud 2.12.0 hef10d8f_5 conda-forge
libgrpc 1.60.1 h74775cd_0 conda-forge
libhwloc 2.9.3 default_h554bfaf_1009 conda-forge
libiconv 1.17 hd590300_2 conda-forge
libidn2 2.3.7 hd590300_0 conda-forge
libjpeg-turbo 3.0.0 hd590300_1 conda-forge
libkvikio 24.04.00a cuda11_240212_g9ef242e_6 rapidsai-nightly
liblapack 3.9.0 21_linux64_openblas conda-forge
libllvm14 14.0.6 hcd5def8_4 conda-forge
libllvm15 15.0.7 hb3ce162_4 conda-forge
libnghttp2 1.58.0 h47da74e_1 conda-forge
libnl 3.9.0 hd590300_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libnuma 2.0.16 h0b41bf4_1 conda-forge
libopenblas 0.3.26 pthreads_h413a1c8_0 conda-forge
libparquet 14.0.2 h352af49_5_cpu conda-forge
libpng 1.6.42 h2797004_0 conda-forge
libprotobuf 4.25.1 hf27288f_1 conda-forge
libraft 24.04.00a55 cuda11_240215_gaddb485f_55 rapidsai-nightly
libraft-headers 24.04.00a55 cuda11_240215_gaddb485f_55 rapidsai-nightly
libraft-headers-only 24.04.00a55 cuda11_240215_gaddb485f_55 rapidsai-nightly
libre2-11 2023.06.02 h7a70373_0 conda-forge
librmm 24.04.00a20 cuda11_240215_gce3af2c4_20 rapidsai-nightly
librsvg 2.56.3 he3f83f7_1 conda-forge
libsanitizer 11.4.0 h4dcbe23_5 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.45.1 h2797004_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx-devel_linux-64 11.4.0 h922705a_105 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge
libthrift 0.19.0 hb90f79a_1 conda-forge
libtiff 4.6.0 ha9c0a0a_2 conda-forge
libunistring 0.9.10 h7f98852_0 conda-forge
libutf8proc 2.8.0 h166bdaf_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libuv 1.47.0 hd590300_0 conda-forge
libwebp 1.3.2 h658648e_1 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libwholegraph 24.04.00a18 cuda11_240215_g3abc3a5_18 rapidsai-nightly
libxcb 1.15 h0b41bf4_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxml2 2.12.5 h232c23b_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
llvmlite 0.42.0 py310h1b8f574_1 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lz4 4.3.3 py310h350c4a5_0 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
markdown 3.5.2 pyhd8ed1ab_0 conda-forge
markdown-it-py 3.0.0 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.5 py310h2372a71_0 conda-forge
matplotlib-base 3.8.2 py310h62c0568_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mdurl 0.1.2 pyhd8ed1ab_0 conda-forge
mistune 3.0.2 pyhd8ed1ab_0 conda-forge
mpi 1.0 openmpi conda-forge
msgpack-python 1.0.7 py310hd41b1e2_0 conda-forge
multidict 6.0.5 py310h2372a71_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nbclient 0.8.0 pyhd8ed1ab_0 conda-forge
nbconvert 7.16.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.16.0 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 7.16.0 pyhd8ed1ab_0 conda-forge
nbformat 5.9.2 pyhd8ed1ab_0 conda-forge
nbsphinx 0.9.3 pyhd8ed1ab_0 conda-forge
nccl 2.20.3.1 h6103f9b_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge
networkx 3.2.1 pyhd8ed1ab_0 conda-forge
ninja 1.11.1 h924138e_0 conda-forge
nodeenv 1.8.0 pyhd8ed1ab_0 conda-forge
notebook 7.1.0 pyhd8ed1ab_0 conda-forge
notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge
numba 0.59.0 py310h7dc5dd1_1 conda-forge
numpy 1.26.4 py310hb13e2d6_0 conda-forge
numpydoc 1.6.0 pyhd8ed1ab_0 conda-forge
nvcc_linux-64 11.8 h41dc85b_23 conda-forge
nvcomp 3.0.5 h838ba91_0 conda-forge
nvtx 0.2.8 py310h2372a71_1 conda-forge
openjpeg 2.5.0 h488ebb8_3 conda-forge
openmpi 5.0.1 h4970cb7_101 conda-forge
openssl 3.2.1 hd590300_0 conda-forge
orc 1.9.2 h7829240_1 conda-forge
overrides 7.7.0 pyhd8ed1ab_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.4 py310hcc13569_0 conda-forge
pandoc 3.1.11.1 ha770c72_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
pango 1.50.14 ha41ecd1_2 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.4.1 pyhd8ed1ab_0 conda-forge
pathspec 0.12.1 pyhd8ed1ab_0 conda-forge
pcre2 10.42 hcad00b1_0 conda-forge
pexpect 4.9.0 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.2.0 py310h01dd4db_0 conda-forge
pip 24.0 pyhd8ed1ab_0 conda-forge
pixman 0.43.2 h59595ed_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge
plotly 5.19.0 pyhd8ed1ab_0 conda-forge
pluggy 1.4.0 pyhd8ed1ab_0 conda-forge
pre-commit 3.6.1 pyha770c72_0 conda-forge
prometheus_client 0.19.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.42 pyha770c72_0 conda-forge
protobuf 4.25.1 py310h620c231_0 conda-forge
psutil 5.9.8 py310h2372a71_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptxcompiler 0.8.1 py310h70a93da_2 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
py-cpuinfo 9.0.0 pyhd8ed1ab_0 conda-forge
pyarrow 14.0.2 py310hf9e7431_5_cpu conda-forge
pyarrow-hotfix 0.6 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pydata-sphinx-theme 0.15.2 pyhd8ed1ab_0 conda-forge
pygments 2.17.2 pyhd8ed1ab_0 conda-forge
pylibcugraph 24.04.00a43 cuda11_py310_240212_g38a8cdbb3_43 rapidsai-nightly
pylibcugraphops 24.04.00a12 cuda11_py310_240214_ge65dfa63_12 rapidsai-nightly
pylibraft 24.04.00a55 cuda11_py310_240215_gaddb485f_55 rapidsai-nightly
pylibwholegraph 24.04.00a18 cuda11_py310_240212_g3abc3a5_18 rapidsai-nightly
pynvml 11.4.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyproject-metadata 0.7.1 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytest 8.0.0 pyhd8ed1ab_0 conda-forge
pytest-benchmark 4.0.0 pyhd8ed1ab_0 conda-forge
pytest-cov 4.1.0 pyhd8ed1ab_0 conda-forge
pytest-mpl 0.16.1 pyhd8ed1ab_0 conda-forge
pytest-xdist 3.5.0 pyhd8ed1ab_0 conda-forge
python 3.10.13 hd12c33a_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.19.1 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
python-louvain 0.16 pyhd8ed1ab_0 conda-forge
python-tzdata 2024.1 pyhd8ed1ab_0 conda-forge
python_abi 3.10 4_cp310 conda-forge
pytz 2024.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0.1 py310h2372a71_1 conda-forge
pyzmq 25.1.2 py310h795f18f_0 conda-forge
raft-dask 24.04.00a55 cuda11_py310_240215_gaddb485f_55 rapidsai-nightly
rapids-dask-dependency 24.04.00a2 0 rapidsai-nightly
rdma-core 50.0 hd3aeb46_0 conda-forge
re2 2023.06.02 h2873b5e_0 conda-forge
readline 8.2 h8228510_1 conda-forge
recommonmark 0.7.1 pyhd8ed1ab_0 conda-forge
referencing 0.33.0 pyhd8ed1ab_0 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
rhash 1.4.4 hd590300_0 conda-forge
rich 13.7.0 pyhd8ed1ab_0 conda-forge
rmm 24.04.00a20 cuda11_py310_240215_gce3af2c4_20 rapidsai-nightly
rpds-py 0.17.1 py310hcb5633a_0 conda-forge
s2n 1.4.3 h06160fa_0 conda-forge
scikit-build-core 0.8.1 pyh4af843d_0 conda-forge
scikit-learn 1.4.0 py310h1fdf081_0 conda-forge
scipy 1.12.0 py310hb13e2d6_2 conda-forge
sed 4.8 he412f7d_0 conda-forge
send2trash 1.8.2 pyh41d4057_0 conda-forge
setuptools 69.0.3 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 h9fff704_0 conda-forge
sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
snowballstemmer 2.2.0 pyhd8ed1ab_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
spdlog 1.12.0 hd2e6256_2 conda-forge
sphinx 5.3.0 pyhd8ed1ab_0 conda-forge
sphinx-copybutton 0.5.2 pyhd8ed1ab_0 conda-forge
sphinx-markdown-tables 0.0.17 pyh6c4a22f_0 conda-forge
sphinxcontrib-applehelp 1.0.8 pyhd8ed1ab_0 conda-forge
sphinxcontrib-devhelp 1.0.6 pyhd8ed1ab_0 conda-forge
sphinxcontrib-htmlhelp 2.0.5 pyhd8ed1ab_0 conda-forge
sphinxcontrib-jsmath 1.0.1 pyhd8ed1ab_0 conda-forge
sphinxcontrib-qthelp 1.0.7 pyhd8ed1ab_0 conda-forge
sphinxcontrib-serializinghtml 1.1.10 pyhd8ed1ab_0 conda-forge
sphinxcontrib-websupport 1.2.7 pyhd8ed1ab_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
sysroot_linux-64 2.17 h4a8ded7_13 conda-forge
tabulate 0.9.0 pyhd8ed1ab_1 conda-forge
tblib 3.0.0 pyhd8ed1ab_0 conda-forge
tenacity 8.2.3 pyhd8ed1ab_0 conda-forge
terminado 0.18.0 pyh0d859eb_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.1 pyhd8ed1ab_0 conda-forge
tornado 6.3.3 py310h2372a71_1 conda-forge
traitlets 5.14.1 pyhd8ed1ab_0 conda-forge
types-python-dateutil 2.8.19.20240106 pyhd8ed1ab_0 conda-forge
typing-extensions 4.9.0 hd8ed1ab_0 conda-forge
typing_extensions 4.9.0 pyha770c72_0 conda-forge
typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge
tzdata 2024a h0c530f3_0 conda-forge
ucx 1.15.0 h75e419f_3 conda-forge
ucx-proc 1.0.0 gpu rapidsai
ucx-py 0.37.00a7 py310_240212_ge3d71a4_7 rapidsai-nightly
ukkonen 1.0.1 py310hd41b1e2_4 conda-forge
unicodedata2 15.1.0 py310h2372a71_0 conda-forge
uri-template 1.3.0 pyhd8ed1ab_0 conda-forge
urllib3 2.2.0 pyhd8ed1ab_0 conda-forge
virtualenv 20.25.0 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge
webcolors 1.13 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
websocket-client 1.7.0 pyhd8ed1ab_0 conda-forge
wget 1.20.3 ha35d2d1_1 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.7 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xyzservices 2023.10.1 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
yarl 1.9.4 py310h2372a71_0 conda-forge
zeromq 4.3.5 h59595ed_0 conda-forge
zict 3.0.0 pyhd8ed1ab_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
</pre></details>
Other/Misc.
No response
Code of Conduct
- [X] I agree to follow cuGraph's Code of Conduct
- [X] I have searched the open bugs and have found no duplicates for this bug report
What's the minimum scale (i.e. # GPUs) to reproduce this? Can you reproduce this with 2 GPUs?
What's the minimum scale (i.e. # GPUs) to reproduce this? Can you reproduce this with 2 GPUs?
It's reproducible with 2-GPUs. This was the result from running on a lab machine.
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/karate.csv-directed:True-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/karate.csv-directed:True-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/karate.csv-directed:False-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/karate.csv-directed:False-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:True-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:True-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:True-seeds:[0, 2]-radius:3] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:False-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:False-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/dolphins.csv-directed:False-seeds:[0, 2]-radius:3] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:True-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:True-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:True-seeds:[0, 2]-radius:3] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:False-seeds:[0, 2]-radius:1] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:False-seeds:[0, 2]-radius:2] - AssertionError: ColumnBase are different
FAILED cugraph/cugraph/tests/sampling/test_egonet_mg.py::test_dask_mg_ego_graphs[graph_file:/home/nfs/ralphl/fix-tests/nv-rliu/datasets/email-Eu-core.csv-directed:False-seeds:[0, 2]-radius:3] - AssertionError: ColumnBase are different
=============================================== 16 failed, 56 passed, 363 warnings in 159.28s (0:02:39) ================================================```
After looking at a trend of 1 Node 8-GPU runs across multiple days, it appears that the failure is transient.
You mean this is a heisenbug? Sounds worse but let me try to reproduce this first.
How often can you reproduce this? (Say run test_egonet_mg.py 10 times, how many times you see at least one failure?)
I am running this on my local system with 2 GPUs, and I can't reproduce the test failure. Let me try this on a DGX node as well.