warp icon indicating copy to clipboard operation
warp copied to clipboard
Published 1 week ago • verified publisher icon NVIDIA

Address unnecessary compilations in `test_fem.py`

Open shi-eric opened this issue 3 months ago • 4 comments

Running the test (in serial) takes 883 seconds on my computer.

This test creates 431 directories in the kernel cache, and five of them are for the test_fem.py module itself, which is particularly problematic since it takes 89 seconds for the first compilation of the module for the GPU

Part of the issue is a file with #define WP_TILE_BLOCK_DIM 1 being compiled for GPU devices, which changes the hash.

Full log:

Warp 1.10.0.dev0 initialized:
   CUDA Toolkit 13.0, Driver 13.0
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition" (95 GiB, sm_120, mempool enabled)
   Kernel cache:
     /home/eshi/.cache/warp/1.10.0.dev0
test_adaptive_nanogrid_cuda_0 (__main__.TestFem.test_adaptive_nanogrid_cuda_0) ... Module warp.fem.adaptivity 5c7aaa5 load on device 'cuda:0' took 4411.36 ms  (compiled)
Module warp.fem.geometry.adaptive_nanogrid 026d7e8 load on device 'cuda:0' took 1307.97 ms  (compiled)
Module warp.fem.utils 3d55542 load on device 'cuda:0' took 310.24 ms  (compiled)
Module __main__._test_geo_cells__itp_domainAdaptiveNanogrid_Cell_e353c513 7b94684 load on device 'cuda:0' took 251.35 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainAdaptiveNanogrid_Cell_d830fbdd c1f92fe load on device 'cuda:0' took 1262.38 ms  (compiled)
Module __main__._test_geo_sides__itp_domainAdaptiveNanogrid_Side_2ecc8e70 691d467 load on device 'cuda:0' took 620.73 ms  (compiled)
Module __main__._test_side_normals__itp_domainAdaptiveNanogrid_Side_47078329 5d88433 load on device 'cuda:0' took 311.75 ms  (compiled)
Module warp.fem.adaptivity._count_refined_voxels__itp_fieldImplicit_AdaptiveNanog_0d245651 4ce6d38 load on device 'cuda:0' took 2503.39 ms  (compiled)
ok
test_closest_point_queries (__main__.TestFem.test_closest_point_queries) ... Module __main__ 454a239 load on device 'cuda:0' took 89004.15 ms  (compiled)
ok
test_deformed_geometry_codimensional_cpu (__main__.TestFem.test_deformed_geometry_codimensional_cpu) ... Module warp.fem.utils 57b239e load on device 'cpu' took 1661.24 ms  (compiled)
Module warp.fem.geometry.trimesh 100ca46 load on device 'cpu' took 1926.25 ms  (compiled)
Module warp.fem.space.trimesh_function_space bdc4a8d load on device 'cpu' took 1516.10 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh3D_Cells_Trimesh3D_Tri_P1_a2fd10fc dc9a894 load on device 'cpu' took 1456.25 ms  (compiled)
Module warp.fem.space.restriction 5ae5c3b load on device 'cpu' took 1423.57 ms  (compiled)
Module __main__._rigid_deformation_field__itp_domainTrimesh3D_Cells_Trime_278d62d7 2226202 load on device 'cpu' took 2094.61 ms  (compiled)
Module __main__ f831972 load on device 'cpu' took 5862.23 ms  (compiled)
ok
test_deformed_geometry_codimensional_cuda_0 (__main__.TestFem.test_deformed_geometry_codimensional_cuda_0) ... Module warp.fem.geometry.trimesh a767eb5 load on device 'cuda:0' took 823.09 ms  (compiled)
Module warp.fem.space.trimesh_function_space db5043b load on device 'cuda:0' took 277.03 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh3D_Cells_Trimesh3D_Tri_P1_a2fd10fc 0bbb458 load on device 'cuda:0' took 236.94 ms  (compiled)
Module warp.fem.space.restriction d687b32 load on device 'cuda:0' took 230.83 ms  (compiled)
Module __main__._rigid_deformation_field__itp_domainTrimesh3D_Cells_Trime_1867df41 5fd570f load on device 'cuda:0' took 1103.48 ms  (compiled)
Module __main__ 2b80b78 load on device 'cuda:0' took 98076.96 ms  (compiled)
ok
test_deformed_geometry_cpu (__main__.TestFem.test_deformed_geometry_cpu) ... Module warp.fem.geometry.tetmesh 8b6b357 load on device 'cpu' took 2547.35 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_Tet_P2_Who_c4367751 fcb1431 load on device 'cpu' took 1548.08 ms  (compiled)
Module __main__._rigid_deformation_field__itp_domainTetmesh_Cells_Tetmesh_db343859 5ef9bb8 load on device 'cpu' took 2126.99 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_DefGeo_NodalField_Tetmesh_Tet_P2_52009766 20fec94 load on device 'cpu' took 1976.59 ms  (compiled)
Module __main__._test_geo_cells__itp_domainDefGeo_NodalField_Tet_8ef280c7 7e4aa69 load on device 'cpu' took 1696.88 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_0f54a1c3 a19427c load on device 'cpu' took 2084.63 ms  (compiled)
Module __main__._test_geo_sides__itp_domainDefGeo_NodalField_Tet_1dd842f1 25f8d33 load on device 'cpu' took 2293.56 ms  (compiled)
Module __main__._test_side_normals__itp_domainDefGeo_NodalField_Tet_7f402994 bbc347e load on device 'cpu' took 1995.88 ms  (compiled)
Module __main__ 1767598 load on device 'cpu' took 6886.90 ms  (compiled)
ok
test_deformed_geometry_cuda_0 (__main__.TestFem.test_deformed_geometry_cuda_0) ... Module warp.fem.geometry.tetmesh f560e73 load on device 'cuda:0' took 1120.34 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_Tet_P2_Who_c4367751 49f0480 load on device 'cuda:0' took 261.88 ms  (compiled)
Module __main__._rigid_deformation_field__itp_domainTetmesh_Cells_Tetmesh_70fd6b35 344207b load on device 'cuda:0' took 1313.75 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_DefGeo_NodalField_Tetmesh_Tet_P2_52009766 634de34 load on device 'cuda:0' took 1051.89 ms  (compiled)
Module __main__._test_geo_cells__itp_domainDefGeo_NodalField_Tet_339344d7 6397adf load on device 'cuda:0' took 457.09 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_47c26135 b15bc5d load on device 'cuda:0' took 8203.52 ms  (compiled)
Module __main__._test_geo_sides__itp_domainDefGeo_NodalField_Tet_eed2f276 6410524 load on device 'cuda:0' took 3231.02 ms  (compiled)
Module __main__._test_side_normals__itp_domainDefGeo_NodalField_Tet_3a33f978 3f8a230 load on device 'cuda:0' took 878.44 ms  (compiled)
Module __main__ 43664ad load on device 'cuda:0' took 108379.30 ms  (compiled)
ok
test_dof_mapper (__main__.TestFem.test_dof_mapper) ... ok
test_grad_decomposition_cpu (__main__.TestFem.test_grad_decomposition_cpu) ... Module __main__.grad_decomposition__itg_f8f8_uNodalField_GridTrip_6d42afa4 af1c384 load on device 'cpu' took 2632.64 ms  (compiled)
ok
test_grad_decomposition_cuda_0 (__main__.TestFem.test_grad_decomposition_cuda_0) ... Module __main__.grad_decomposition__itg_f8f8_uNodalField_GridTrip_6d42afa4 4816a73 load on device 'cuda:0' took 7767.67 ms  (compiled)
ok
test_grid_2d_cpu (__main__.TestFem.test_grid_2d_cpu) ... Module __main__._test_geo_cells__itp_domainGrid2D_Cells_Grid2D_C_7cb1db99 9459e15 load on device 'cpu' took 1604.75 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainGrid2D_Cells_Grid2D_C_b16ac6bf f02a066 load on device 'cpu' took 1699.20 ms  (compiled)
Module __main__._test_geo_sides__itp_domainGrid2D_Sides_Grid2D_S_08246e0f bb3672d load on device 'cpu' took 1751.91 ms  (compiled)
Module __main__._test_side_normals__itp_domainGrid2D_Sides_Grid2D_S_a541105e 7ccd5f4 load on device 'cpu' took 1539.62 ms  (compiled)
ok
test_grid_2d_cuda_0 (__main__.TestFem.test_grid_2d_cuda_0) ... Module __main__._test_geo_cells__itp_domainGrid2D_Cells_Grid2D_C_7cb1db99 ae4638e load on device 'cuda:0' took 251.18 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainGrid2D_Cells_Grid2D_C_b16ac6bf 0fcc240 load on device 'cuda:0' took 420.90 ms  (compiled)
Module __main__._test_geo_sides__itp_domainGrid2D_Sides_Grid2D_S_08246e0f 45d3402 load on device 'cuda:0' took 482.11 ms  (compiled)
Module __main__._test_side_normals__itp_domainGrid2D_Sides_Grid2D_S_a541105e 3e132b6 load on device 'cuda:0' took 410.94 ms  (compiled)
ok
test_grid_3d_cpu (__main__.TestFem.test_grid_3d_cpu) ... Module __main__._test_geo_cells__itp_domainGrid3D_Cells_Grid3D_C_f512e73f 1192edd load on device 'cpu' took 1549.11 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainGrid3D_Cells_Grid3D_C_5e8766e3 7546a34 load on device 'cpu' took 1730.47 ms  (compiled)
Module __main__._test_geo_sides__itp_domainGrid3D_Sides_Grid3D_S_56caf396 b3907af load on device 'cpu' took 1761.50 ms  (compiled)
Module __main__._test_side_normals__itp_domainGrid3D_Sides_Grid3D_S_fb768a66 c85583a load on device 'cpu' took 1609.18 ms  (compiled)
ok
test_grid_3d_cuda_0 (__main__.TestFem.test_grid_3d_cuda_0) ... Module __main__._test_geo_cells__itp_domainGrid3D_Cells_Grid3D_C_f512e73f a6df7d9 load on device 'cuda:0' took 283.52 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainGrid3D_Cells_Grid3D_C_5e8766e3 2c681cd load on device 'cuda:0' took 467.06 ms  (compiled)
Module __main__._test_geo_sides__itp_domainGrid3D_Sides_Grid3D_S_56caf396 54a1727 load on device 'cuda:0' took 583.57 ms  (compiled)
Module __main__._test_side_normals__itp_domainGrid3D_Sides_Grid3D_S_fb768a66 312421c load on device 'cuda:0' took 321.37 ms  (compiled)
ok
test_hex_mesh_cpu (__main__.TestFem.test_hex_mesh_cpu) ... Module warp.fem.geometry.hexmesh 16552d4 load on device 'cpu' took 2871.35 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Hexmesh_8c75c3b1 3bbde81 load on device 'cpu' took 1610.37 ms  (compiled)
Module __main__._test_geo_cells__itp_domainHexmesh_Cells_Hexmesh_3f1d5de3 5fdd54d load on device 'cpu' took 1589.46 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainHexmesh_Cells_Hexmesh_bcf9685d e5b851a load on device 'cpu' took 1844.37 ms  (compiled)
Module __main__._test_geo_sides__itp_domainHexmesh_Sides_Hexmesh_60cbba71 c70be07 load on device 'cpu' took 1892.21 ms  (compiled)
Module __main__._test_side_normals__itp_domainHexmesh_Sides_Hexmesh_fe3535f0 36f4336 load on device 'cpu' took 1541.97 ms  (compiled)
ok
test_hex_mesh_cuda_0 (__main__.TestFem.test_hex_mesh_cuda_0) ... Module warp.fem.geometry.hexmesh 5dbcd55 load on device 'cuda:0' took 1895.64 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Hexmesh_8c75c3b1 3db2b61 load on device 'cuda:0' took 439.78 ms  (compiled)
Module __main__._test_geo_cells__itp_domainHexmesh_Cells_Hexmesh_3f1d5de3 443a399 load on device 'cuda:0' took 347.01 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainHexmesh_Cells_Hexmesh_bcf9685d 53c9b40 load on device 'cuda:0' took 2389.70 ms  (compiled)
Module __main__._test_geo_sides__itp_domainHexmesh_Sides_Hexmesh_60cbba71 34dd5fa load on device 'cuda:0' took 900.34 ms  (compiled)
Module __main__._test_side_normals__itp_domainHexmesh_Sides_Hexmesh_fe3535f0 8d923b6 load on device 'cuda:0' took 333.81 ms  (compiled)
ok
test_implicit_fields (__main__.TestFem.test_implicit_fields) ... Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_GridBipolynomialSpa_a398f513 9d76318 load on device 'cuda:0' took 248.12 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldUniformGrid2D_Cells_f4_53a8b6ee b26563a load on device 'cuda:0' took 539.79 ms  (compiled)
Module __main__.grad_field__itp_pUniformGrid2D_Cells_f4_Gri_a61366ee 01a132d load on device 'cuda:0' took 541.47 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_BoundarySides_GridBipolyn_758802fe 37530e7 load on device 'cuda:0' took 314.16 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldUniformGrid2D_Sides_f4_7c249f87 9c4f9d6 load on device 'cuda:0' took 925.28 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldImplicit_Grid2D_Cells__31898af6 0fea270 load on device 'cuda:0' took 481.03 ms  (compiled)
Module __main__.grad_field__itp_pImplicit_Grid2D_Cells_0_Ar_28b663b2 5bf7c2e load on device 'cuda:0' took 518.00 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldImplicit_Grid2D_Sides__44513a82 42835cc load on device 'cuda:0' took 1179.36 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldGrid2D_Cells_NodalFiel_979cbda8 eb79c29 load on device 'cuda:0' took 765.18 ms  (compiled)
Module __main__.grad_field__itp_pGrid2D_Cells_NodalField_Gr_a72307c8 76a3552 load on device 'cuda:0' took 958.78 ms  (compiled)
Module warp.fem.integrate._identity_field__itp_fieldGrid2D_Sides_NodalFiel_36afdbd9 257c0fe load on device 'cuda:0' took 1526.51 ms  (compiled)
ok
test_integrate_gradient_cpu (__main__.TestFem.test_integrate_gradient_cpu) ... Module __main__.linear_form__itg_f4f8_uNodalField_GridBipo_9c55af51 c9fc692 load on device 'cpu' took 1842.88 ms  (compiled)
Module warp.utils 946ec48 load on device 'cpu' took 1629.78 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_GridBipolynomialSpa_cc8b4146 89dbf3d load on device 'cpu' took 1524.91 ms  (compiled)
Module __main__.scaled_linear_form__itg_f4f8_uTestFieldGridBipoly_301118fd 94d01ea load on device 'cpu' took 1839.44 ms  (compiled)
Module __main__.scaled_linear_form__itg_f4f8_uTestFieldGridBipoly_3d9cdee5 9e79e0e load on device 'cpu' took 1609.21 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldGridBipolynomialSpaceTo_b9982eba cf8acd4 load on device 'cpu' took 1757.68 ms  (compiled)
ok
test_integrate_gradient_cuda_0 (__main__.TestFem.test_integrate_gradient_cuda_0) ... Module __main__.linear_form__itg_f4f8_uNodalField_GridBipo_98e5674b c0fd713 load on device 'cuda:0' took 593.63 ms  (compiled)
Module warp.utils 5e751fc load on device 'cuda:0' took 795.10 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_GridBipolynomialSpa_cc8b4146 629d27a load on device 'cuda:0' took 266.80 ms  (compiled)
Module __main__.scaled_linear_form__itg_f4f8_uTestFieldGridBipoly_dd634cc4 a6d003c load on device 'cuda:0' took 547.90 ms  (compiled)
Module __main__.scaled_linear_form__itg_f4f8_uTestFieldGridBipoly_b72421dd 64ff327 load on device 'cuda:0' took 330.05 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldGridBipolynomialSpaceTo_b9982eba c91af7c load on device 'cuda:0' took 419.23 ms  (compiled)
ok
test_integrate_high_order_cuda_0 (__main__.TestFem.test_integrate_high_order_cuda_0) ... Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid3D_Cells_GridTripolynomialSp_70c2ca50 1d045a8 load on device 'cuda:0' took 314.74 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldGridTripol_14ce5bc6 af34959 load on device 'cuda:0' took 263.95 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldGridTripolynomialSpaceT_39445e83 23fc291 load on device 'cuda:0' took 296.63 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldGridTripol_bbaa792e 96b9af5 load on device 'cuda:0' took 301.93 ms  (compiled)
Module __main__.bilinear_form__itg_f8f8_uTrialFieldGridTripo_58d60c42 78bd090 load on device 'cuda:0' took 268.50 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_bilinear_kernel_fn_TrialFieldGridTripolynomialSpace_232ce62d 7f263b5 load on device 'cuda:0' took 390.13 ms  (compiled)
Module warp.sparse 627f70e load on device 'cuda:0' took 671.94 ms  (compiled)
Module __main__.bilinear_form__itg_f8f8_uTrialFieldGridTripo_279eba14 75dc4f1 load on device 'cuda:0' took 336.07 ms  (compiled)
ok
test_interpolate_gradient_cpu (__main__.TestFem.test_interpolate_gradient_cpu) ... Module warp.fem.geometry.quadmesh fa10589 load on device 'cpu' took 1893.87 ms  (compiled)
Module warp.fem.space.quadmesh_function_space 354eaae load on device 'cpu' took 1494.83 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_3971dfd4 2f1139f load on device 'cpu' took 1523.40 ms  (compiled)
Module __main__.bilinear_field__itp_domainQuadmesh2D_Cells_Quad_65fed4ba 5320007 load on device 'cpu' took 2004.08 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_PointTopology_E_4a09b490 eece58d load on device 'cpu' took 1464.18 ms  (compiled)
Module __main__.grad_field__itp_pNodalField_Quadmesh2D_Squa_4b231e97 6272c99 load on device 'cpu' took 2336.34 ms  (compiled)
Module __main__.grad_field__itp_pTrialFieldQuadmesh2D_Squar_47453450 624585e load on device 'cpu' took 1712.95 ms  (compiled)
Module warp.sparse b8af487 load on device 'cpu' took 1680.77 ms  (compiled)
Module warp.sparse.dyn.bsr_mv_kernel_1_7cc665ae e2bb1b1 load on device 'cpu' took 1437.04 ms  (compiled)
ok
test_interpolate_gradient_cuda_0 (__main__.TestFem.test_interpolate_gradient_cuda_0) ... Module warp.fem.geometry.quadmesh 0a79320 load on device 'cuda:0' took 1048.84 ms  (compiled)
Module warp.fem.space.quadmesh_function_space 805f4c6 load on device 'cuda:0' took 295.98 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_3971dfd4 7202a29 load on device 'cuda:0' took 275.38 ms  (compiled)
Module __main__.bilinear_field__itp_domainQuadmesh2D_Cells_Quad_65fed4ba 18fa4bd load on device 'cuda:0' took 926.73 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_PointTopology_E_4a09b490 3298b6a load on device 'cuda:0' took 340.39 ms  (compiled)
Module __main__.grad_field__itp_pNodalField_Quadmesh2D_Squa_4b231e97 e1fa8b0 load on device 'cuda:0' took 2175.65 ms  (compiled)
Module __main__.grad_field__itp_pTrialFieldQuadmesh2D_Squar_47453450 b5f191a load on device 'cuda:0' took 384.67 ms  (compiled)
Module warp.sparse 4fba51c load on device 'cuda:0' took 954.26 ms  (compiled)
Module warp.sparse.dyn.bsr_mv_kernel_1_7cc665ae b643289 load on device 'cuda:0' took 389.95 ms  (compiled)
ok
test_nanogrid_cuda_0 (__main__.TestFem.test_nanogrid_cuda_0) ... Module warp.fem.geometry.nanogrid 3e6edc8 load on device 'cuda:0' took 639.16 ms  (compiled)
Module __main__._test_geo_cells__itp_domainNanogrid_Cells_Nanogr_78092839 2ae3ca0 load on device 'cuda:0' took 252.27 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainNanogrid_Cells_Nanogr_013ae728 f0bd789 load on device 'cuda:0' took 1060.29 ms  (compiled)
Module __main__._test_geo_sides__itp_domainNanogrid_Sides_Nanogr_0262fb22 8c7f45d load on device 'cuda:0' took 658.27 ms  (compiled)
Module __main__._test_side_normals__itp_domainNanogrid_Sides_Nanogr_57518d12 9f9f84a load on device 'cuda:0' took 315.55 ms  (compiled)
ok
test_nodal_quadrature (__main__.TestFem.test_nodal_quadrature) ... Module warp.fem.quadrature.quadrature.dyn.quadrature_point_element_indices_NodalQuadrature_Grid2D_D9_Square_1a65b12a e239df1 load on device 'cuda:0' took 255.50 ms  (compiled)
Module __main__._bicubic__itg_f8f8_domainGrid2D_CellsNo_ad6e1ee2 e8805a2 load on device 'cuda:0' took 423.31 ms  (compiled)
Module warp.fem.geometry.partition.dyn.count_sides_Grid2D_LinearGeometryPartition___058e7f23 4c1fc47 load on device 'cuda:0' took 303.32 ms  (compiled)
Module warp.fem.space.partition.dyn.node_category_from_cells_kernel_Grid2D_LinearGeometryPartition_G_9f051b85 c610a24 load on device 'cuda:0' took 245.68 ms  (compiled)
Module warp.fem.space.partition 81e1b67 load on device 'cuda:0' took 249.19 ms  (compiled)
Module warp.fem.quadrature.quadrature.dyn.quadrature_point_element_indices_NodalQuadrature_GridBipolynomial_5a7b1d2e 3f89755 load on device 'cuda:0' took 243.90 ms  (compiled)
Module __main__._value_at_node__itg_f8f8_fNodalField_GridBipo_baa63a7f 93ac382 load on device 'cuda:0' took 459.93 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_Grid2D_D9_Whole_fd359115 3dc765c load on device 'cuda:0' took 283.68 ms  (compiled)
Module __main__._test_node_index__itg_f8f8_uTestFieldGrid2D_D9__24027a40 d560e86 load on device 'cuda:0' took 282.32 ms  (compiled)
ok
test_particle_quadratures (__main__.TestFem.test_particle_quadratures) ... Module __main__._bicubic__itg_f8f8_domainGrid2D_CellsEx_328360c3 d6c0678 load on device 'cuda:0' took 451.89 ms  (compiled)
Module __main__._piecewise_constant__itp__Grid2D_Cells_ExplicitQuadr_f19732a4 89d8083 load on device 'cuda:0' took 338.52 ms  (compiled)
Module warp.fem.quadrature.pic_quadrature 87526f6 load on device 'cuda:0' took 317.47 ms  (compiled)
Module warp.fem.quadrature.quadrature.dyn.quadrature_point_element_indices_PicQuadratureGrid2D_Cells_09303573 a95f4be load on device 'cuda:0' took 490.16 ms  (compiled)
Module __main__._piecewise_constant__itg_f8f8_PicQuadrature_edcf3e9d a8fb4fd load on device 'cuda:0' took 392.19 ms  (compiled)
Module warp.fem.quadrature.pic_quadrature.dyn.bin_particles_Grid2D_Cells_3fcf88f2 142221e load on device 'cuda:0' took 595.27 ms  (compiled)
Module warp.fem.quadrature.pic_quadrature.dyn.compute_fraction_Grid2D_Cells_efb9b648 2f6ef59 load on device 'cuda:0' took 298.84 ms  (compiled)
ok
test_point_basis (__main__.TestFem.test_point_basis) ... Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_PointTopology_Regul_bcc58ada 5e5e6be load on device 'cuda:0' took 234.75 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldRegularQua_964ddc08 81536c9 load on device 'cuda:0' took 576.01 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldRegularQua_c6b89b17 bb43533 load on device 'cuda:0' took 453.05 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldRegularQuadrature_Grid2_9e40313d d6996ff load on device 'cuda:0' took 475.22 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Grid2D_Cells_PointTopology_PicQu_437e5f79 acfd53e load on device 'cuda:0' took 340.60 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldPicQuadrat_0b077314 1c302a9 load on device 'cuda:0' took 364.90 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldPicQuadrature_Point_flo_fb894304 f589b8a load on device 'cuda:0' took 431.20 ms  (compiled)
Module __main__.linear_form__itg_f8f8_uTestFieldPicQuadrat_3ce9e992 825580d load on device 'cuda:0' took 360.85 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_linear_kernel_fn_TestFieldPicQuadrature_Point_flo_6e1468e1 6106bb3 load on device 'cuda:0' took 406.23 ms  (compiled)
Module __main__.vector_divergence_form__itg_f8f8_uTestFieldGridBipoly_f460f60a ff88534 load on device 'cuda:0' took 467.00 ms  (compiled)
Module warp.fem.field.virtual.dyn.dispatch_bilinear_kernel_fn_TrialFieldPicQuadrature_Point_fl_6905e931 acac4c4 load on device 'cuda:0' took 549.08 ms  (compiled)
ok
test_quad_mesh_cpu (__main__.TestFem.test_quad_mesh_cpu) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Quadmesh2D_51ea2cb2 d59dcba load on device 'cpu' took 1676.36 ms  (compiled)
Module __main__._test_geo_cells__itp_domainQuadmesh2D_Cells_Quad_a5ec3a58 2633a18 load on device 'cpu' took 1601.56 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainQuadmesh2D_Cells_Quad_0d62cc77 0cb5fc0 load on device 'cpu' took 2071.17 ms  (compiled)
Module __main__._test_geo_sides__itp_domainQuadmesh2D_Sides_Quad_c6a50329 951f780 load on device 'cpu' took 1807.13 ms  (compiled)
Module __main__._test_side_normals__itp_domainQuadmesh2D_Sides_Quad_40619e19 1d7f07b load on device 'cpu' took 1771.90 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Quadmesh3D_fdc9a7b6 ee771b4 load on device 'cpu' took 1802.94 ms  (compiled)
Module __main__._test_geo_cells__itp_domainQuadmesh3D_Cells_Quad_2d6007e8 1cfdd79 load on device 'cpu' took 1729.06 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainQuadmesh3D_Cells_Quad_9f207796 9f98e03 load on device 'cpu' took 1972.41 ms  (compiled)
Module __main__._test_geo_sides__itp_domainQuadmesh3D_Sides_Quad_a554d38c a42c470 load on device 'cpu' took 1837.79 ms  (compiled)
ok
test_quad_mesh_cuda_0 (__main__.TestFem.test_quad_mesh_cuda_0) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Quadmesh2D_51ea2cb2 4f8876e load on device 'cuda:0' took 338.08 ms  (compiled)
Module __main__._test_geo_cells__itp_domainQuadmesh2D_Cells_Quad_a5ec3a58 3e26460 load on device 'cuda:0' took 313.07 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainQuadmesh2D_Cells_Quad_0d62cc77 abd4557 load on device 'cuda:0' took 1736.85 ms  (compiled)
Module __main__._test_geo_sides__itp_domainQuadmesh2D_Sides_Quad_c6a50329 20abbf1 load on device 'cuda:0' took 640.32 ms  (compiled)
Module __main__._test_side_normals__itp_domainQuadmesh2D_Sides_Quad_40619e19 c899868 load on device 'cuda:0' took 332.51 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Quadmesh3D_fdc9a7b6 04535f7 load on device 'cuda:0' took 346.18 ms  (compiled)
Module __main__._test_geo_cells__itp_domainQuadmesh3D_Cells_Quad_2d6007e8 ea50e0f load on device 'cuda:0' took 302.79 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainQuadmesh3D_Cells_Quad_9f207796 762560d load on device 'cuda:0' took 1856.72 ms  (compiled)
Module __main__._test_geo_sides__itp_domainQuadmesh3D_Sides_Quad_a554d38c 6839981 load on device 'cuda:0' took 633.21 ms  (compiled)
ok
test_regular_quadrature (__main__.TestFem.test_regular_quadrature) ... ok
test_tensor_divergence_theorem_cpu (__main__.TestFem.test_tensor_divergence_theorem_cpu) ... Module __main__.tensor_divergence_form__itg_f8f8_tauNodalField_GridBi_12986cd9 031d48e load on device 'cpu' took 1717.18 ms  (compiled)
Module __main__.tensor_boundary_form__itg_f8f8_tauNodalFieldTrace__f173b1e8 09ebf91 load on device 'cpu' took 1928.72 ms  (compiled)
Module __main__.tensor_divergence_form__itg_f8f8_tauNodalField_GridBi_e99a4aa9 11ce551 load on device 'cpu' took 1938.38 ms  (compiled)
Module __main__.tensor_grad_form__itg_f8f8_tauNodalField_GridBi_697f750a 6a7b038 load on device 'cpu' took 1867.72 ms  (compiled)
Module __main__.tensor_boundary_form__itg_f8f8_tauNodalFieldTrace__1dec0a81 aa1af7f load on device 'cpu' took 1961.61 ms  (compiled)
ok
test_tensor_divergence_theorem_cuda_0 (__main__.TestFem.test_tensor_divergence_theorem_cuda_0) ... Module __main__.tensor_divergence_form__itg_f8f8_tauNodalField_GridBi_12986cd9 6cd1eef load on device 'cuda:0' took 456.54 ms  (compiled)
Module __main__.tensor_boundary_form__itg_f8f8_tauNodalFieldTrace__f173b1e8 352fe82 load on device 'cuda:0' took 646.88 ms  (compiled)
Module __main__.tensor_divergence_form__itg_f8f8_tauNodalField_GridBi_e99a4aa9 0ef6540 load on device 'cuda:0' took 498.39 ms  (compiled)
Module __main__.tensor_grad_form__itg_f8f8_tauNodalField_GridBi_697f750a aa3ea3f load on device 'cuda:0' took 489.47 ms  (compiled)
Module __main__.tensor_boundary_form__itg_f8f8_tauNodalFieldTrace__1dec0a81 a9a0465 load on device 'cuda:0' took 673.02 ms  (compiled)
ok
test_tet_mesh_cpu (__main__.TestFem.test_tet_mesh_cpu) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Tetmesh_f0f002c9 2ecdd7f load on device 'cpu' took 1586.69 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTetmesh_Cells_Tetmesh_8d89d439 ba00dd7 load on device 'cpu' took 1726.13 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTetmesh_Cells_Tetmesh_6dfe8ada 9c1f6d1 load on device 'cpu' took 1930.21 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTetmesh_Sides_Tetmesh_7882a100 b038271 load on device 'cpu' took 2024.15 ms  (compiled)
Module __main__._test_side_normals__itp_domainTetmesh_Sides_Tetmesh_8b2831ba 9610411 load on device 'cpu' took 1589.24 ms  (compiled)
ok
test_tet_mesh_cuda_0 (__main__.TestFem.test_tet_mesh_cuda_0) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Tetmesh_f0f002c9 ecffb85 load on device 'cuda:0' took 340.06 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTetmesh_Cells_Tetmesh_8d89d439 002c311 load on device 'cuda:0' took 296.55 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTetmesh_Cells_Tetmesh_6dfe8ada 5524477 load on device 'cuda:0' took 1579.78 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTetmesh_Sides_Tetmesh_7882a100 5b524da load on device 'cuda:0' took 739.12 ms  (compiled)
Module __main__._test_side_normals__itp_domainTetmesh_Sides_Tetmesh_8b2831ba c1c2506 load on device 'cuda:0' took 306.55 ms  (compiled)
ok
test_triangle_mesh_cpu (__main__.TestFem.test_triangle_mesh_cpu) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Trimesh2D_5e06569d e5d75c3 load on device 'cpu' took 1534.41 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTrimesh2D_Cells_Trime_085d16f2 6ad1e2e load on device 'cpu' took 1534.17 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTrimesh2D_Cells_Trime_36d42325 b149f87 load on device 'cpu' took 1832.75 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTrimesh2D_Sides_Trime_4f36f2f6 9d81ca2 load on device 'cpu' took 1887.74 ms  (compiled)
Module __main__._test_side_normals__itp_domainTrimesh2D_Sides_Trime_13a6259a 0cd46c1 load on device 'cpu' took 1599.63 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Trimesh3D_27c8b693 80aec87 load on device 'cpu' took 1607.67 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTrimesh3D_Cells_Trime_d2125adb 31a42f0 load on device 'cpu' took 1642.70 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTrimesh3D_Cells_Trime_66b35c86 4be4daf load on device 'cpu' took 1871.80 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTrimesh3D_Sides_Trime_32d819b4 fadb0f1 load on device 'cpu' took 1780.67 ms  (compiled)
ok
test_triangle_mesh_cuda_0 (__main__.TestFem.test_triangle_mesh_cuda_0) ... Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Trimesh2D_5e06569d 2787213 load on device 'cuda:0' took 328.21 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTrimesh2D_Cells_Trime_085d16f2 c75095b load on device 'cuda:0' took 291.24 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTrimesh2D_Cells_Trime_36d42325 fe08fa9 load on device 'cuda:0' took 978.06 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTrimesh2D_Sides_Trime_4f36f2f6 f9412c3 load on device 'cuda:0' took 555.07 ms  (compiled)
Module __main__._test_side_normals__itp_domainTrimesh2D_Sides_Trime_13a6259a 9175af9 load on device 'cuda:0' took 334.30 ms  (compiled)
Module warp.fem.geometry.geometry.dyn.compute_cell_bounds_Trimesh3D_27c8b693 8d7fe56 load on device 'cuda:0' took 334.11 ms  (compiled)
Module __main__._test_geo_cells__itp_domainTrimesh3D_Cells_Trime_d2125adb 76913cf load on device 'cuda:0' took 274.93 ms  (compiled)
Module __main__._test_cell_lookup__itp_domainTrimesh3D_Cells_Trime_66b35c86 d2c4d8d load on device 'cuda:0' took 1046.69 ms  (compiled)
Module __main__._test_geo_sides__itp_domainTrimesh3D_Sides_Trime_32d819b4 40e2523 load on device 'cuda:0' took 627.03 ms  (compiled)
ok
test_vector_divergence_theorem_cpu (__main__.TestFem.test_vector_divergence_theorem_cpu) ... Module __main__.vector_divergence_form__itg_f8f8_uNodalField_GridBipo_b1207669 d19beb2 load on device 'cpu' took 1766.98 ms  (compiled)
Module __main__.vector_boundary_form__itg_f8f8_uNodalFieldTrace_Gr_2185a5a0 3d83c85 load on device 'cpu' took 1835.43 ms  (compiled)
Module __main__.vector_divergence_form__itg_f8f8_uNodalField_GridBipo_5aa99744 0141577 load on device 'cpu' took 1673.91 ms  (compiled)
Module __main__.vector_grad_form__itg_f8f8_uNodalField_GridBipo_5019714f 7d7ec81 load on device 'cpu' took 1679.76 ms  (compiled)
Module __main__.vector_boundary_form__itg_f8f8_uNodalFieldTrace_Gr_e1e2a06a a28ce03 load on device 'cpu' took 1831.33 ms  (compiled)
ok
test_vector_divergence_theorem_cuda_0 (__main__.TestFem.test_vector_divergence_theorem_cuda_0) ... Module __main__.vector_divergence_form__itg_f8f8_uNodalField_GridBipo_b1207669 dbb8429 load on device 'cuda:0' took 426.88 ms  (compiled)
Module __main__.vector_boundary_form__itg_f8f8_uNodalFieldTrace_Gr_2185a5a0 22adf72 load on device 'cuda:0' took 592.17 ms  (compiled)
Module __main__.vector_divergence_form__itg_f8f8_uNodalField_GridBipo_5aa99744 d5f6fba load on device 'cuda:0' took 397.96 ms  (compiled)
Module __main__.vector_grad_form__itg_f8f8_uNodalField_GridBipo_5019714f f295434 load on device 'cuda:0' took 393.50 ms  (compiled)
Module __main__.vector_boundary_form__itg_f8f8_uNodalFieldTrace_Gr_e1e2a06a fe924d8 load on device 'cuda:0' took 567.46 ms  (compiled)
ok
test_vector_spaces_cpu (__main__.TestFem.test_vector_spaces_cpu) ... Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_b0032535 9a53c27 load on device 'cpu' took 1450.91 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Quadm_0cca9f7f b78de39 load on device 'cpu' took 2642.63 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_78685fa4 225e5a2 load on device 'cpu' took 1509.04 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Quadm_5f372571 875f6e0 load on device 'cpu' took 2582.62 ms  (compiled)
Module warp.fem.space.hexmesh_function_space 0b0e7cf load on device 'cpu' took 1454.14 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Hexmesh_Cells_Hexmesh_CubeN1_1_W_5a69337b d00baad load on device 'cpu' took 1534.37 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Hexme_ab11e6de 3559c1d load on device 'cpu' took 2807.36 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Hexmesh_Cells_Hexmesh_CubeRT_1_W_ee201789 6c314a5 load on device 'cpu' took 1453.82 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Hexme_0a5d5ed5 c2ffc79 load on device 'cpu' took 2610.42 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh2D_Cells_Trimesh2D_TriN1__52a89813 6ba9a7e load on device 'cpu' took 1607.50 ms  (compiled)
Module __main__._expect_pure_curl__itg_f8f8_fieldTestFieldTrimes_e3e9ae13 0d8a3a5 load on device 'cpu' took 2236.04 ms  (compiled)
Module __main__._expect_pure_curl__itp_fieldNodalField_Trimesh2D_T_7403fcc1 baa099c load on device 'cpu' took 2073.83 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Trime_834e4285 78c43e0 load on device 'cpu' took 2624.78 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh2D_Cells_Trimesh2D_TriRT__f5253abe db18849 load on device 'cpu' took 1457.22 ms  (compiled)
Module __main__._expect_pure_spherical__itg_f8f8_fieldTestFieldTrimes_115d76ec 51f60de load on device 'cpu' took 1974.50 ms  (compiled)
Module __main__._expect_pure_spherical__itp_fieldNodalField_Trimesh2D_T_172c7a70 0ea210f load on device 'cpu' took 1975.07 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Trime_ab8bb203 73ab0a7 load on device 'cpu' took 2514.13 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_TetN1_1_Wh_8e53fca8 9ee3dff load on device 'cpu' took 1460.44 ms  (compiled)
Module __main__._expect_pure_curl__itg_f8f8_fieldTestFieldTetmes_729b0db8 d02beaf load on device 'cpu' took 2039.92 ms  (compiled)
Module __main__._expect_pure_curl__itp_fieldNodalField_Tetmesh_Tet_a5b89583 881d517 load on device 'cpu' took 2060.71 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Tetme_8de2bc9a 6a4d3f3 load on device 'cpu' took 2539.02 ms  (compiled)
Module warp.fem.space.tetmesh_function_space 8ee0f7d load on device 'cpu' took 1531.08 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_TetRT_1_Wh_219da047 d67ec80 load on device 'cpu' took 1448.13 ms  (compiled)
Module __main__._expect_pure_spherical__itg_f8f8_fieldTestFieldTetmes_1c98e3ee 9481b07 load on device 'cpu' took 2063.90 ms  (compiled)
Module __main__._expect_pure_spherical__itp_fieldNodalField_Tetmesh_Tet_738dbacc 30f21fd load on device 'cpu' took 2236.72 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Tetme_42497587 d3df78d load on device 'cpu' took 2548.60 ms  (compiled)
ok
test_vector_spaces_cuda_0 (__main__.TestFem.test_vector_spaces_cuda_0) ... Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_b0032535 a98a871 load on device 'cuda:0' took 247.77 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Quadm_0cca9f7f f442f40 load on device 'cuda:0' took 13680.85 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Quadmesh2D_Cells_Quadmesh2D_Squa_78685fa4 9d69244 load on device 'cuda:0' took 245.33 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Quadm_5f372571 4f9f2af load on device 'cuda:0' took 11372.26 ms  (compiled)
Module warp.fem.space.hexmesh_function_space f1c8332 load on device 'cuda:0' took 302.78 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Hexmesh_Cells_Hexmesh_CubeN1_1_W_5a69337b 112937d load on device 'cuda:0' took 274.93 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Hexme_ab11e6de fb8c537 load on device 'cuda:0' took 20260.40 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Hexmesh_Cells_Hexmesh_CubeRT_1_W_ee201789 91aaac6 load on device 'cuda:0' took 242.76 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Hexme_0a5d5ed5 b2adf15 load on device 'cuda:0' took 13084.85 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh2D_Cells_Trimesh2D_TriN1__52a89813 72dd589 load on device 'cuda:0' took 244.01 ms  (compiled)
Module __main__._expect_pure_curl__itg_f8f8_fieldTestFieldTrimes_e3e9ae13 e92d669 load on device 'cuda:0' took 1442.43 ms  (compiled)
Module __main__._expect_pure_curl__itp_fieldNodalField_Trimesh2D_T_7403fcc1 f9b5009 load on device 'cuda:0' took 2188.36 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Trime_834e4285 fb7ab17 load on device 'cuda:0' took 11258.90 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Trimesh2D_Cells_Trimesh2D_TriRT__f5253abe 509f9b3 load on device 'cuda:0' took 242.68 ms  (compiled)
Module __main__._expect_pure_spherical__itg_f8f8_fieldTestFieldTrimes_115d76ec 1d4d7a7 load on device 'cuda:0' took 1201.78 ms  (compiled)
Module __main__._expect_pure_spherical__itp_fieldNodalField_Trimesh2D_T_172c7a70 ac9f6a4 load on device 'cuda:0' took 1759.04 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Trime_ab8bb203 df9afca load on device 'cuda:0' took 9263.65 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_TetN1_1_Wh_8e53fca8 883a1e0 load on device 'cuda:0' took 265.40 ms  (compiled)
Module __main__._expect_pure_curl__itg_f8f8_fieldTestFieldTetmes_729b0db8 0aae5b5 load on device 'cuda:0' took 1921.40 ms  (compiled)
Module __main__._expect_pure_curl__itp_fieldNodalField_Tetmesh_Tet_a5b89583 0f75e79 load on device 'cuda:0' took 2874.74 ms  (compiled)
Module __main__._expect_tangential_continuity__itp_fieldNodalFieldTrace_Tetme_8de2bc9a 13d5c09 load on device 'cuda:0' took 14737.13 ms  (compiled)
Module warp.fem.space.tetmesh_function_space 7ce13fd load on device 'cuda:0' took 321.10 ms  (compiled)
Module warp.fem.space.restriction.dyn.fill_element_node_indices_Tetmesh_Cells_Tetmesh_TetRT_1_Wh_219da047 93631c9 load on device 'cuda:0' took 242.98 ms  (compiled)
Module __main__._expect_pure_spherical__itg_f8f8_fieldTestFieldTetmes_1c98e3ee 1a4fea3 load on device 'cuda:0' took 1495.56 ms  (compiled)
Module __main__._expect_pure_spherical__itp_fieldNodalField_Tetmesh_Tet_738dbacc e5c034f load on device 'cuda:0' took 1965.26 ms  (compiled)
Module __main__._expect_normal_continuity__itp_fieldNodalFieldTrace_Tetme_42497587 92f7ad7 load on device 'cuda:0' took 11828.53 ms  (compiled)
ok
test_cube_shape_functions (__main__.TestFemShapeFunctions.test_cube_shape_functions) ... Module __main__.dyn.node_unity_test_Cube_Q1_LGL_16519d63 60c188b load on device 'cuda:0' took 259.00 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q1_LGL_96c87da7 f8c6b99 load on device 'cuda:0' took 223.29 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q1_LGL_56489c2c 725c4c4 load on device 'cuda:0' took 236.44 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_Q2_LGL_d3486504 a37ed25 load on device 'cuda:0' took 280.27 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q2_LGL_eab90f86 53db3df load on device 'cuda:0' took 249.56 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q2_LGL_a9a504a7 2f7254b load on device 'cuda:0' took 254.80 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_Q3_LGL_0a5ee369 d46557d load on device 'cuda:0' took 243.55 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q3_LGL_5c362c7e 68a6284 load on device 'cuda:0' took 222.01 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q3_LGL_e58f028e 4a4d9f8 load on device 'cuda:0' took 285.92 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Cube_Q1_LGL_8fca10bd b173182 load on device 'cuda:0' took 230.75 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Cube_Q2_LGL_191cbebe 56051f9 load on device 'cuda:0' took 242.34 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Cube_Q3_LGL_6bd5ee0c 1feb43c load on device 'cuda:0' took 244.69 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q1_LGL_23042fc7 193bb05 load on device 'cuda:0' took 254.57 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q2_LGL_3fe235d1 4651cee load on device 'cuda:0' took 299.99 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q3_LGL_ee3fcc83 ea82205 load on device 'cuda:0' took 316.73 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_Q1_GL_69573970 5b4ecf2 load on device 'cuda:0' took 271.31 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q1_GL_aced107a 32c723b load on device 'cuda:0' took 220.96 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q1_GL_00e4d279 13f7a2c load on device 'cuda:0' took 240.68 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_Q2_GL_d279c5d7 30e069c load on device 'cuda:0' took 278.68 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q2_GL_83a49f7a 139f6b0 load on device 'cuda:0' took 247.11 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q2_GL_252f91e7 1f8bb13 load on device 'cuda:0' took 252.40 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_Q3_GL_042edcee 53338e1 load on device 'cuda:0' took 240.53 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_Q3_GL_4df301f9 223e62d load on device 'cuda:0' took 220.09 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_Q3_GL_c56fb64b 8a796b6 load on device 'cuda:0' took 285.19 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q1_GL_d72f264a dbd89ef load on device 'cuda:0' took 285.88 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q2_GL_1e93ff9a e132a64 load on device 'cuda:0' took 300.60 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_Q3_GL_5f8d32e3 2bb3035 load on device 'cuda:0' took 317.29 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_S2_LGL_beb25c48 913f9b5 load on device 'cuda:0' took 263.96 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_S2_LGL_3219b770 4335b78 load on device 'cuda:0' took 264.63 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_S2_LGL_449bc7a7 778897e load on device 'cuda:0' took 269.07 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_S3_LGL_c830f6da f0209d6 load on device 'cuda:0' took 264.51 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_S3_LGL_8e7c3bee 90246c4 load on device 'cuda:0' took 280.37 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_S3_LGL_c4ce5fb8 a5c247c load on device 'cuda:0' took 283.44 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Cube_S2_LGL_933116b2 ee8b942 load on device 'cuda:0' took 263.69 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Cube_S3_LGL_c81456b0 cbc01e9 load on device 'cuda:0' took 286.25 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_S2_LGL_5c280cf8 32b4c07 load on device 'cuda:0' took 329.45 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_S3_LGL_4d1bcdb6 3b3face load on device 'cuda:0' took 338.04 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_P1d_c0108200 44e7b1a load on device 'cuda:0' took 244.77 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_P1d_b31d69b6 957abba load on device 'cuda:0' took 250.25 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_P1d_c2c947b8 194f5e3 load on device 'cuda:0' took 261.19 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_P2d_e1e6756b 7612e99 load on device 'cuda:0' took 258.60 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_P2d_4c289787 25f67c4 load on device 'cuda:0' took 343.44 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_P2d_2eb43982 b46298a load on device 'cuda:0' took 325.93 ms  (compiled)
Module __main__.dyn.node_unity_test_Cube_P3d_97db3d42 bfa6531 load on device 'cuda:0' took 283.40 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Cube_P3d_2cdf8fdb 3cc5dd9 load on device 'cuda:0' took 275.54 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Cube_P3d_3c34db0b 1f085e4 load on device 'cuda:0' took 266.85 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_P1d_12955a57 c0d03fa load on device 'cuda:0' took 286.19 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_P2d_91e45546 8fee217 load on device 'cuda:0' took 314.79 ms  (compiled)
Module __main__.dyn.finite_difference_test_Cube_P3d_c8b8e5b9 61fd71d load on device 'cuda:0' took 347.39 ms  (compiled)
Module __main__.dyn.finite_difference_test_CubeN1_1_041a02b8 ac4315a load on device 'cuda:0' took 551.80 ms  (compiled)
Module __main__.dyn.finite_difference_test_CubeRT_1_9841cacf ce89254 load on device 'cuda:0' took 270.33 ms  (compiled)
ok
test_square_shape_functions (__main__.TestFemShapeFunctions.test_square_shape_functions) ... Module __main__.dyn.node_unity_test_Square_Q1_LGL_32b00de5 acae006 load on device 'cuda:0' took 251.30 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q1_LGL_d8d81c6d 511a0f5 load on device 'cuda:0' took 211.87 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q1_LGL_4f1a348d 954722a load on device 'cuda:0' took 256.36 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_Q2_LGL_1d57483b 2ebd8f8 load on device 'cuda:0' took 265.81 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q2_LGL_8193d85f c962bdb load on device 'cuda:0' took 224.68 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q2_LGL_36cf302f 5cfb8fd load on device 'cuda:0' took 241.66 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_Q3_LGL_d107125f 231fb15 load on device 'cuda:0' took 270.36 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q3_LGL_bdbd8f8c c22f7b9 load on device 'cuda:0' took 243.48 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q3_LGL_d1d8972c cba7a3a load on device 'cuda:0' took 254.59 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Square_Q1_LGL_2f0e9da1 c492477 load on device 'cuda:0' took 215.81 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Square_Q2_LGL_a0e7dc03 1d040db load on device 'cuda:0' took 233.87 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Square_Q3_LGL_02516bc1 20e5fa5 load on device 'cuda:0' took 260.17 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q1_LGL_1a89142f f848b86 load on device 'cuda:0' took 246.96 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q2_LGL_ce65710b 9789f9c load on device 'cuda:0' took 280.08 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q3_LGL_7a512f8b 7208303 load on device 'cuda:0' took 289.25 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_Q1_GL_6a1242fd a594611 load on device 'cuda:0' took 258.96 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q1_GL_19b2168f adc3c87 load on device 'cuda:0' took 212.05 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q1_GL_cbbce097 4bec41e load on device 'cuda:0' took 232.29 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_Q2_GL_2462ee64 449d596 load on device 'cuda:0' took 265.30 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q2_GL_74159329 8913498 load on device 'cuda:0' took 226.36 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q2_GL_8321859c 5cd30b5 load on device 'cuda:0' took 241.60 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_Q3_GL_e3a41299 162909d load on device 'cuda:0' took 270.85 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_Q3_GL_09b5e146 077c514 load on device 'cuda:0' took 243.71 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_Q3_GL_54422421 ad36b30 load on device 'cuda:0' took 253.87 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q1_GL_f7b03cb6 611196d load on device 'cuda:0' took 269.75 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q2_GL_23d43e43 c7bbcf9 load on device 'cuda:0' took 281.37 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_Q3_GL_5932235d c9fde69 load on device 'cuda:0' took 290.00 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_S2_LGL_677ec2b4 5e4a7e6 load on device 'cuda:0' took 297.92 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_S2_LGL_b159e400 b03788c load on device 'cuda:0' took 226.32 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_S2_LGL_f36f324a 872fbb0 load on device 'cuda:0' took 251.21 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_S3_LGL_9d873776 b909ceb load on device 'cuda:0' took 284.95 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_S3_LGL_78d51f29 888dac5 load on device 'cuda:0' took 234.42 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_S3_LGL_84a549e7 326a810 load on device 'cuda:0' took 258.93 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Square_S2_LGL_8c30f0cc fe42cb4 load on device 'cuda:0' took 234.85 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Square_S3_LGL_97667447 798a8f9 load on device 'cuda:0' took 249.99 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_S2_LGL_c14afab3 9e33100 load on device 'cuda:0' took 363.49 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_S3_LGL_91901911 1146204 load on device 'cuda:0' took 357.73 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_P1d_cfe0bec0 792f507 load on device 'cuda:0' took 238.43 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_P1d_515b2c69 e1970b8 load on device 'cuda:0' took 233.29 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_P1d_eb4ef076 4eb3e0b load on device 'cuda:0' took 245.69 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_P2d_28e1d02e f3b16f9 load on device 'cuda:0' took 245.01 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_P2d_caea8990 7ffff8e load on device 'cuda:0' took 262.19 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_P2d_feb99e58 f0bb881 load on device 'cuda:0' took 264.86 ms  (compiled)
Module __main__.dyn.node_unity_test_Square_P3d_19befe66 a844d4c load on device 'cuda:0' took 259.65 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Square_P3d_cc1f2410 3171f93 load on device 'cuda:0' took 312.53 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Square_P3d_e4b270fd a8354e6 load on device 'cuda:0' took 297.84 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_P1d_078d3f95 65f3a8e load on device 'cuda:0' took 273.95 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_P2d_8e2612ef 8f2be30 load on device 'cuda:0' took 294.49 ms  (compiled)
Module __main__.dyn.finite_difference_test_Square_P3d_78d6bc7c 27e8b38 load on device 'cuda:0' took 344.61 ms  (compiled)
Module __main__.dyn.finite_difference_test_SquareN1_1_28df94eb 3a9481f load on device 'cuda:0' took 270.37 ms  (compiled)
Module __main__.dyn.finite_difference_test_SquareRT_1_e2b157e5 222add0 load on device 'cuda:0' took 334.99 ms  (compiled)
ok
test_tet_shape_functions (__main__.TestFemShapeFunctions.test_tet_shape_functions) ... Module __main__.dyn.node_unity_test_Tet_P1_c48c0eb8 ba28301 load on device 'cuda:0' took 377.50 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P1_c3dd7176 2297962 load on device 'cuda:0' took 256.34 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P1_efad9d98 44eacad load on device 'cuda:0' took 331.14 ms  (compiled)
Module __main__.dyn.node_unity_test_Tet_P2_8ab4e43d cc0091a load on device 'cuda:0' took 307.32 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P2_88f4fbd3 abf9b92 load on device 'cuda:0' took 242.24 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P2_815e6cca f821acb load on device 'cuda:0' took 269.79 ms  (compiled)
Module __main__.dyn.node_unity_test_Tet_P3_77b08d77 a8e4575 load on device 'cuda:0' took 264.51 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P3_54279f64 da622db load on device 'cuda:0' took 241.19 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P3_46ea0995 0757c13 load on device 'cuda:0' took 257.97 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tet_P1_69f6b422 910193f load on device 'cuda:0' took 227.33 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tet_P2_5a3de93b 0212202 load on device 'cuda:0' took 256.81 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tet_P3_89d395ce 8eb6a13 load on device 'cuda:0' took 248.51 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P1_5e074469 48c5cd2 load on device 'cuda:0' took 272.44 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P2_e2b1c2d0 195b8ce load on device 'cuda:0' took 287.13 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P3_1e95f722 d05c3a0 load on device 'cuda:0' took 412.09 ms  (compiled)
Module __main__.dyn.node_unity_test_Tet_P1d_48908ed4 422e1ee load on device 'cuda:0' took 288.40 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P1d_d24b2f75 99b1457 load on device 'cuda:0' took 229.89 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P1d_4613d94f 1b1b5c0 load on device 'cuda:0' took 287.56 ms  (compiled)
Module __main__.dyn.node_unity_test_Tet_P2d_0b7d99ca b42376d load on device 'cuda:0' took 262.66 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P2d_c7392010 c1ea5dc load on device 'cuda:0' took 252.26 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P2d_7a994c22 8ad29e1 load on device 'cuda:0' took 283.83 ms  (compiled)
Module __main__.dyn.node_unity_test_Tet_P3d_6d7ee987 c3a9b0c load on device 'cuda:0' took 278.62 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tet_P3d_52c4e168 31f4c28 load on device 'cuda:0' took 246.20 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tet_P3d_1d13205b fff5ca5 load on device 'cuda:0' took 265.33 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P1d_78f01e97 2e8e7f6 load on device 'cuda:0' took 270.64 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P2d_964a111b df9b0f4 load on device 'cuda:0' took 298.58 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tet_P3d_7f37442e 12c289c load on device 'cuda:0' took 330.05 ms  (compiled)
Module __main__.dyn.finite_difference_test_TetN1_1_ce5152e2 4fc6088 load on device 'cuda:0' took 293.67 ms  (compiled)
Module __main__.dyn.finite_difference_test_TetRT_1_4449db16 d61b255 load on device 'cuda:0' took 275.52 ms  (compiled)
ok
test_tri_shape_functions (__main__.TestFemShapeFunctions.test_tri_shape_functions) ... Module __main__.dyn.node_unity_test_Tri_P1_628072fb bd745be load on device 'cuda:0' took 251.92 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_P1_1d2a777c 21e4ffb load on device 'cuda:0' took 217.52 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_P1_3e8d4424 5aabdc5 load on device 'cuda:0' took 250.30 ms  (compiled)
Module __main__.dyn.node_unity_test_Tri_P2_ec40b53f 9fa39e3 load on device 'cuda:0' took 268.83 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_P2_aade48de 37be342 load on device 'cuda:0' took 226.70 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_P2_cfe10a88 ceff479 load on device 'cuda:0' took 257.99 ms  (compiled)
Module __main__.dyn.node_unity_test_Tri_P3_86471075 2edd2d4 load on device 'cuda:0' took 300.41 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_P3_c97aa75a 2856675 load on device 'cuda:0' took 241.44 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_P3_e1dc9bd9 d2d6e98 load on device 'cuda:0' took 268.17 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tri_P1_17eccbf8 22ad8b6 load on device 'cuda:0' took 247.57 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tri_P2_6f158799 000df26 load on device 'cuda:0' took 244.92 ms  (compiled)
Module __main__.dyn.trace_node_quadrature_unity_test_Tri_P3_0ccb5382 e1deb2f load on device 'cuda:0' took 249.91 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_P1_898dc74b 304189a load on device 'cuda:0' took 247.43 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_P2_8449db86 f218499 load on device 'cuda:0' took 269.80 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_P3_c5f37e68 7092747 load on device 'cuda:0' took 287.76 ms  (compiled)
Module __main__.dyn.node_unity_test_Tri_dP1_b0ea6dbb 52818d4 load on device 'cuda:0' took 236.72 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_dP1_d90818ac 5116397 load on device 'cuda:0' took 236.21 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_dP1_024bbb9c 4fe3a49 load on device 'cuda:0' took 245.93 ms  (compiled)
Module __main__.dyn.node_unity_test_Tri_dP2_0b12878e 1d8f1ca load on device 'cuda:0' took 245.23 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_dP2_a34ee577 812c407 load on device 'cuda:0' took 266.17 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_dP2_14fab556 b1f8d87 load on device 'cuda:0' took 266.47 ms  (compiled)
Module __main__.dyn.node_unity_test_Tri_dP3_52005037 5be90b8 load on device 'cuda:0' took 259.02 ms  (compiled)
Module __main__.dyn.node_quadrature_unity_test_Tri_dP3_2cb9b592 24ae0a6 load on device 'cuda:0' took 321.93 ms  (compiled)
Module __main__.dyn.partition_of_unity_test_Tri_dP3_d5e38156 723db6b load on device 'cuda:0' took 302.82 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_dP1_23e855b0 5d95053 load on device 'cuda:0' took 267.98 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_dP2_0121d162 b2cd7e9 load on device 'cuda:0' took 288.95 ms  (compiled)
Module __main__.dyn.finite_difference_test_Tri_dP3_12096723 8595697 load on device 'cuda:0' took 311.01 ms  (compiled)
Module __main__.dyn.finite_difference_test_TriN1_1_7c2d8979 63c2cd3 load on device 'cuda:0' took 269.77 ms  (compiled)
ok
test_array_axpy (__main__.TestFemUtilities.test_array_axpy) ... Module warp.fem.linalg 108a633 load on device 'cuda:0' took 225.68 ms  (compiled)
Module warp.fem.linalg faa83ce load on device 'cuda:0' took 253.14 ms  (compiled)
Module warp.fem.linalg 02da00b load on device 'cuda:0' took 280.78 ms  (compiled)
ok
test_qr_eigenvalues_cpu (__main__.TestFemUtilities.test_qr_eigenvalues_cpu) ... Module __main__ 43664ad load on device 'cpu' took 6802.75 ms  (compiled)
ok
test_qr_eigenvalues_cuda_0 (__main__.TestFemUtilities.test_qr_eigenvalues_cuda_0) ... Module __main__ 1767598 load on device 'cuda:0' took 106927.72 ms  (compiled)
ok
test_qr_inverse_cpu (__main__.TestFemUtilities.test_qr_inverse_cpu) ... ok
test_qr_inverse_cuda_0 (__main__.TestFemUtilities.test_qr_inverse_cuda_0) ... ok

----------------------------------------------------------------------
Ran 47 tests in 883.792s

OK

shi-eric avatar Sep 28 '25 19:09 shi-eric

There are a couple of dynamic kernels on main that can should moved to their own modules, I'll do that. But the compile time is surprising -- I get about 17s on my machine, wonder if this is a CUDA arch thing?

gdaviet avatar Sep 30 '25 09:09 gdaviet

64b85f91c6ea75723c867bba5cb4b083653ac8fa reduced the time to run the test by about 300 secs

Leaving a note for the future: On my system there were some remaining modules that took a long time to compile. One noticeable one was (Using Warp v1.9.0):

Run 1

Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_0f54a1c3 2b967c1 load on device 'cuda:0' took 49214.32 ms  (compiled)

Run 2

Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_0f54a1c3 2b967c1 load on device 'cuda:0' took 50645.80 ms  (compiled)

This file has about 3800 lines.

shi-eric avatar Sep 30 '25 16:09 shi-eric

For reference the same kernel takes about 5s to compile on my machine (CUDA 13.0 as well, but sm_86)

Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_0f54a1c3 2b967c1 load on device 'cuda:0' took 5536.38 ms  (compiled)

Would be interesting to know why there's such a large difference

gdaviet avatar Oct 01 '25 09:10 gdaviet

For reference the same kernel takes about 5s to compile on my machine (CUDA 13.0 as well, but sm_86)

Module __main__._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_0f54a1c3 2b967c1 load on device 'cuda:0' took 5536.38 ms  (compiled)

Would be interesting to know why there's such a large difference

Thanks for the data point. It does seem arch related as the Blackwell GitLab runner running the linux-x86_64-blackwell test job on GitLab also indicates a long compile time:

Module warp.tests.test_fem._test_cell_lookup__itp_domainDefGeo_NodalField_Tet_4923400e b5c5a55 load on device 'cuda:0' took 56401.52 ms  (compiled)

shi-eric avatar Oct 01 '25 16:10 shi-eric