balsa icon indicating copy to clipboard operation
balsa copied to clipboard

The version of cuda

Open Young0222 opened this issue 1 year ago • 0 comments

I chose the 3-rd solution, i.e., (3) Don't visualize my results, to run this code, but met a mistake (maybe the problem about the version of CUDA), the detailed information is as follows:

Traceback (most recent call last):
  File "run.py", line 2157, in <module>
    app.run(Main)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "run.py", line 2153, in Main
    agent.Run()
  File "run.py", line 2102, in Run
    has_timeouts = self.RunOneIter()
  File "run.py", line 1843, in RunOneIter
    is_test=False)
  File "run.py", line 1415, in PlanAndExecute
    avoid_eq_filters=is_test and p.avoid_eq_filters,
File "/home/xjc/balsa/balsa/optimizer.py", line 255, in plan
    **kwargs)
  File "/home/xjc/balsa/balsa/optimizer.py", line 508, in _beam_search_bk
    [join for join, _, _ in possible_plans])
  File "/home/xjc/balsa/balsa/optimizer.py", line 217, in infer
    cost1 = self.value_network(query_feat, plan_feat, pos_feat)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "run.py", line 544, in forward
    return self.model(query_feat, plan_feat, indexes)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xjc/balsa/balsa/models/treeconv.py", line 96, in forward
    query_embs = self.query_mlp(query_feats.unsqueeze(1))
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/container.py", line 100, in forward
    input = module(input)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/xjc/anaconda3/envs/balsa/lib/python3.7/site-packages/torch/nn/functional.py", line 1372, in linear
    output = input.matmul(weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

What should I do to solve it? Thanks a lot!

Young0222 avatar Mar 13 '23 03:03 Young0222