hipCaffe icon indicating copy to clipboard operation
hipCaffe copied to clipboard

build/test/test_all.testbin Core Dump MIOpen Error

Open gateway opened this issue 6 years ago • 1 comments

Please use the caffe-users list for usage, installation, or modeling questions, or other requests for help. Do not post such requests to Issues. Doing so interferes with the development of Caffe.

Please read the guidelines for contributing before submitting this issue.

Issue summary

Fresh install of Ubuntu 16.x Desktop, ROCM and started to build hipCafee however ran into a error when running ./build/test/test_all.testbin

Stack Trace:

[ RUN      ] SoftmaxWithLossLayerTest/2.TestGradientUnnormalized
[       OK ] SoftmaxWithLossLayerTest/2.TestGradientUnnormalized (89 ms)
[----------] 4 tests from SoftmaxWithLossLayerTest/2 (2434 ms total)

[----------] 4 tests from SoftmaxWithLossLayerTest/3, where TypeParam = caffe::GPUDevice<double>
[ RUN      ] SoftmaxWithLossLayerTest/3.TestGradient
MIOpen Error: /home/dlowell/MIOpenPrivate/src/ocl/softmaxocl.cpp:59: Only alpha=1 and beta=0 is supported
F1227 11:47:35.865128 23305 cudnn_softmax_layer_hip.cpp:27] Check failed: status == miopenStatusSuccess (7 vs. 0)  miopenStatusUnknownError
*** Check failure stack trace: ***
    @     0x7f6f6c7d15cd  google::LogMessage::Fail()
    @     0x7f6f6c7d3433  google::LogMessage::SendToLog()
    @     0x7f6f6c7d115b  google::LogMessage::Flush()
    @     0x7f6f6c7d3e1e  google::LogMessageFatal::~LogMessageFatal()
    @          0x14a1f2e  caffe::CuDNNSoftmaxLayer<>::Forward_gpu()
    @           0x4ed860  caffe::Layer<>::Forward()
    @          0x1d9a153  caffe::SoftmaxWithLossLayer<>::Forward_gpu()
    @           0x4ed860  caffe::Layer<>::Forward()
    @           0x51c010  caffe::GradientChecker<>::CheckGradientSingle()
    @           0x51b76d  caffe::GradientChecker<>::CheckGradientExhaustive()
    @           0xa21807  caffe::SoftmaxWithLossLayerTest_TestGradient_Test<>::TestBody()
    @          0x1033034  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @          0x1032ef6  testing::Test::Run()
    @          0x1034051  testing::TestInfo::Run()
    @          0x10348b7  testing::TestCase::Run()
    @          0x103ada7  testing::internal::UnitTestImpl::RunAllTests()
    @          0x103a7e4  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @          0x103a799  testing::UnitTest::Run()
    @          0x1ef98fa  main
    @     0x7f6f6783f830  __libc_start_main
    @          0x1ef8d99  _start
    @              (nil)  (unknown)
Aborted (core dumped)

Scrolling up to see if any errors I did find this , I believe this was on make test.

Expected: data[i]
Which is: 1.6294847
src/caffe/test/test_inner_product_layer.cpp:384: Failure
Value of: data_t[i]
  Actual: 2.9882355
Expected: data[i]
Which is: 2.474798
src/caffe/test/test_inner_product_layer.cpp:384: Failure
Value of: data_t[i]
  Actual: 2.1015618
Expected: data[i]
Which is: 2.0466099

Thoughts, suggestions I could try..

Your system configuration

Operating system: Ubuntu 16 Desktop Compiler: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.11) CUDA version (if applicable): NA CUDNN version (if applicable): BLAS: Python or MATLAB version (for pycaffe and matcaffe respectively):

gateway avatar Dec 27 '18 20:12 gateway

Versions of miopen..

miopen-hip is already the newest version (1.7.0-49c48917).
miopengemm is already the newest version (1.1.5-9547fb9).
miopengemm set to manually installed.
rocm-libs is already the newest version (2.0.89).

gateway avatar Dec 27 '18 20:12 gateway