skip msp verify hulls for testing

[hossam26644]

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.84%. Comparing base (52d3b8d) to head (220af5a). Report is 20 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2359      +/-   ##
==========================================
- Coverage   90.93%   90.84%   -0.09%     
==========================================
  Files          20       20              
  Lines       12095    11978     -117     
  Branches     2325     2325              
==========================================
- Hits        10999    10882     -117     
  Misses        606      606              
  Partials      490      490              

Flag	Coverage Δ
C	90.84% <100.00%> (-0.09%)	:arrow_down:
python	98.70% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeromekelleher]

I think the better approach here is to comment out msp_verify where ever it's being called in this file. This will kill performance on any of the models.

[hossam26644]

@jeromekelleher what about I just add a return statement in the first line of msp verify? msp verify is called many times in the test files; they could be left to run for testing, I can comment all the calls in the test files(i do not like this), or just cancel msp verify by the return statement I suggested.

[jeromekelleher]

This was how it's designed - it should never be called in production code, it's meant to be used during development and called explicitly from the C test suite. So, just comment out/delete whereever it's currently getting called in the production code. It's probably just a few lines where debugging stuff has been left in place.

[jeromekelleher]

Oh, I see - were you running all the evaluations through the dev CLI? Let's not do this, the dev-cli isn't for profiling it's strictly for development (and almost never used now anyway).

So - I don't think we need this change, you just need to stop using the dev CLI and run through Python instead.

[jeromekelleher]

I'm going to close this as the dev cli isn't something we want to use for anything serious