CUDA Runtime Error: an illegal memory access was encountered

[oeway]

Hi, I encountered an error when running apps/single_curve_sdf_trans.py with GPU.

~/diffvg/apps$ CUDA_VISIBLE_DEVICES=1 python single_curve_sdf_trans.py 
CUDA Runtime Error: an illegal memory access was encountered at ~/diffvg/diffvg.cpp:1631

When this error occurred, the terminal hangs.

    // Clean up weight image
    if (scene->use_gpu) {
#ifdef __CUDACC__
        checkCuda(cudaFree(weight_image)); <--------------error occurred to this line
#else
        assert(false);
#endif
    } else {
        free(weight_image);
    }

[BachiLi]

I can't reproduce this. Why do you want to run this particular file? It's just for some random experiments with SDF rendering.

[oeway]

It was just an issue I came across when I was trying this particular file.

[preddy5]

@BachiLi I am encountering the same issue as well, I think it might because of the number of shapes. In my scripts, the same code works as expected for 256 shapes but breaks if I am estimating 289 shapes.

[BachiLi]

is it a GPU specific issue? or does it also happen on CPU? i don't think i make this assumption anywhere in the code

[preddy5]

I have changed all my input variables to the render function to CPU, it still raises the same error if I try to render more than 256 shape.

[BachiLi]

any chance you can build diffvg in debug mode and let me know at which line it breaks?

[preddy5]

I dont know if this helps though, same error on cpu too.

     36 
     37       optim.zero_grad()
---> 38       loss.backward()
     39       optim.step()
     40 

~/anaconda3/envs/autosets/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    116                 products. Defaults to ``False``.
    117         """
--> 118         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    119 
    120     def register_hook(self, hook):

~/anaconda3/envs/autosets/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     91     Variable._execution_engine.run_backward(
     92         tensors, grad_tensors, retain_graph, create_graph,
---> 93         allow_unreachable=True)  # allow_unreachable flag
     94 
     95 

RuntimeError: cuda runtime error (9) : invalid configuration argument at /opt/conda/conda-bld/pytorch_1565272271120/work/aten/src/THC/generic/THCTensorMath.cu:35

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1565272271120/work/aten/src/THC/generic/THCTensorMath.cu line=35 error=9 : invalid configuration argument

[preddy5]

Also the script raises the same error even with fewer shapes in debug mode.

[BachiLi]

Unfortuantely not helpful : ( Can you share which line in the C++ code it breaks in the CPU mode.

[preddy5]

this is the line that the error is raised in normal mode, for more than 256 shapes CUDA Runtime Error: an illegal memory access was encountered at ~/diffvg/diffvg.cpp:1631 However in debug mode diffvg doesn't raise any error. The render returns an image as well, but backward pass raises an error for less than 256 shapes as well.

[BachiLi]

sorry i was in the wrong mindset: diffvg doesn't have CPU mode : ( can you run it with cuda-memcheck?

[preddy5]

So far I have been working with jupyter notebook, when I run the code as a script I have new error, I think this might help,

/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [379,0,0], thread: [17,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [379,0,0], thread: [20,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [379,0,0], thread: [21,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [403,0,0], thread: [16,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [5,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [9,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [10,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [11,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [13,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [14,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [15,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [16,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [17,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [18,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [19,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [20,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [21,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [22,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [23,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [24,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [395,0,0], thread: [25,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [299,0,0], thread: [15,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [5,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [7,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [8,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [9,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [10,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [355,0,0], thread: [17,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [363,0,0], thread: [13,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [363,0,0], thread: [15,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [363,0,0], thread: [18,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [363,0,0], thread: [19,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [13,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [14,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [15,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [16,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [17,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [18,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [19,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [20,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [22,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [371,0,0], thread: [23,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [291,0,0], thread: [18,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [323,0,0], thread: [23,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [323,0,0], thread: [30,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [315,0,0], thread: [10,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [339,0,0], thread: [10,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [339,0,0], thread: [11,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [4,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [5,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [6,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [7,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [9,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [11,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [13,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [14,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [15,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [16,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [18,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [19,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [20,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [21,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [22,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [23,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [24,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [26,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [387,0,0], thread: [27,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [411,0,0], thread: [20,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [411,0,0], thread: [21,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [419,0,0], thread: [11,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [427,0,0], thread: [13,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [347,0,0], thread: [7,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [435,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [443,0,0], thread: [12,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [331,0,0], thread: [27,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [283,0,0], thread: [10,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [307,0,0], thread: [14,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [370,0,0], thread: [61,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [378,0,0], thread: [61,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [354,0,0], thread: [63,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [322,0,0], thread: [62,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [338,0,0], thread: [59,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [346,0,0], thread: [57,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [314,0,0], thread: [62,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [298,0,0], thread: [62,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [418,0,0], thread: [61,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [410,0,0], thread: [61,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [402,0,0], thread: [61,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [306,0,0], thread: [55,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [306,0,0], thread: [57,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [306,0,0], thread: [62,0,0] Assertion `num_fragments < max_hit_shapes` failed.
/home/creddy/Work/diffvg/diffvg.cpp:556: TVector4<float> sample_color(const SceneData &, const TVector4<float> *, const TVector2<float> &, const TVector4<float> *, EdgeQuery *, TVector4<float> *, float *): block: [330,0,0], thread: [59,0,0] Assertion `num_fragments < max_hit_shapes` failed.
CUDA Runtime Error: device-side assert triggered at /home/creddy/Work/diffvg/diffvg.cpp:1594

[BachiLi]

Change max_hit_shapes to a larger number? https://github.com/BachiLi/diffvg/blob/fd38f71527b6258ddaf54ba2db3596b396dc80e1/diffvg.cpp#L542

Sorry this is a stupid restriction. I need to fix it : (

[preddy5]

@BachiLi works like a charm, thank you for being responsive.

[MADONOKOUKI]

Does anyone happen similar situation at backpropagation? (no error on CPU but error on GPU) Ubuntu 18.04 cuda 10.0 cudnn 7.0 It might be due to a large amount of parameters (Just I use one GPU since I can't parallel the GPUs now).

iteration: 0
Scene construction, time: 40.43806 s
Forward pass, time: 0.00457 s
cuda:0 torch.float32 torch.float32 cuda:0
Traceback (most recent call last):
  File "main.py", line 217, in <module>
    loss.backward()
  File "/root/anaconda3/envs/py37/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/root/anaconda3/envs/py37/lib/python3.7/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag
  File "/root/anaconda3/envs/py37/lib/python3.7/site-packages/torch/autograd/function.py", line 77, in apply
    return self._forward_cls.backward(self, *args)
  File "/home/myname/adobe/tmp/meshgrad/pydiffvg/render_pytorch.py", line 556, in backward
    eval_positions.shape[0])
RuntimeError
CUDA Runtime Error: an illegal memory access was encountered at /home/myname/adobe/tmp/meshgrad/scene.cpp:1523