MOE
MOE copied to clipboard
Fix flakey tests, increase tol, p1 or 2 to set threads RNG
moe.tests.optimal_learning.python.cpp_wrappers.expected_improvement_test ExpectedImprovementTest.test_python_and_cpp_return_same_1d_analytic_ei_and_gradient
Traceback (most recent call last):
File "./moe/tests/optimal_learning/python/cpp_wrappers/expected_improvement_test.py", line 87, in test_python_and_cpp_return_same_1d_analytic_ei_and_gradient
self.assert_vector_within_relative(python_grad_ei, cpp_grad_ei, grad_ei_tolerance)
File "./moe/tests/optimal_learning/python/optimal_learning_test_case.py", line 62, in assert_vector_within_relative
self.assert_scalar_within_relative(value[index], truth[index], tol)
File "./moe/tests/optimal_learning/python/optimal_learning_test_case.py", line 41, in assert_scalar_within_relative
message='value = {0:.18E}, truth = {1:.18E}, diff = {2:.18E}, tol = {3:.18E}'.format(value, truth, diff, tol),
AssertionError: value = -1.710635215382066629E-05, truth = -1.710635215381947367E-05, diff = 6.971810114804449073E-14, tol = 5.999999999999999677E-14
FAIL in 0.38s
AND
moe.
tests.optimal_learning.python.cpp_wrappers.gaussian_process_test GaussianProcessTest.test_python_and_cpp_return_same_cholesky_variance_and_gradient
Traceback (most recent call last):
File "./moe/tests/optimal_learning/python/cpp_wrappers/gaussian_process_test.py", line 132, in test_python_and_cpp_return_same_cholesky_variance_and_gradient
self.assert_vector_within_relative(python_grad_var, cpp_grad_var, grad_var_tolerance)
File "./moe/tests/optimal_learning/python/optimal_learning_test_case.py", line 62, in assert_vector_within_relative
self.assert_scalar_within_relative(value[index], truth[index], tol)
File "./moe/tests/optimal_learning/python/optimal_learning_test_case.py", line 41, in assert_scalar_within_relative
message='value = {0:.18E}, truth = {1:.18E}, diff = {2:.18E}, tol = {3:.18E}'.format(value, truth, diff, tol),
AssertionError: value = -1.810166738444277133E-06, truth = -1.810166738419891690E-06, diff = 1.347137960231614092E-11, tol = 3.000000000000000142E-12
FAIL in 1.38s
tolerances decreased so these tests pass. I'm bumping the priority of this ticket down. It is now about making these tests more repeatable (i.e., by setting RNG)