[backend] Refactor api-test folders (#13317)

[aHenryJard]

Proposed changes

• Refactor test folder to be able to run tests that inject data lonely (02)
• Do not require anymore the whole integration test to run rule tests
• Have a pure unit step without any infra
• Moves 02-integration/03-import before resolvers, and fix all required counts

Structure is

tests
   - 01-unit => these test can run without any infra (elastic, redis...)
   - 02-dataInjection => tests that inject data with python + tests that count this data
   - 03-integration => former 02-integration
   - 10-streams
   - 11-sync
   - 20-rules
   - 30-httpTaxii
   - 99-deprecated

New yarn command are mainly used by CI but can also be used locally:

# Run unit tests, no need for elastic or redis
yarn test:ci-unit

# Run data injection +  integration domain & resolvers test, but not sync part
yarn test:ci-integration

# Run data injection + integration domain & resolvers test, including sync part
yarn test:ci-integration-sync

# Run data injection + rule tests
test:ci-rules-and-others

Documentation PR: https://github.com/OpenCTI-Platform/docs/pull/390

Related issues

• closes #13317

Checklist

• [ ] I consider the submitted work as finished
• [ ] I tested the code for its functionality
• [ ] I wrote test cases for the relevant uses case (coverage and e2e)
• [ ] I added/update the relevant documentation (either on github or on notion)
• [ ] Where necessary I refactored code to improve the overall quality

Further comments

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 30.79%. Comparing base (71019ab) to head (4aa1926). :warning: Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #13318   +/-   ##
=======================================
  Coverage   30.79%   30.79%           
=======================================
  Files        2907     2907           
  Lines      192125   192165   +40     
  Branches    39164    39186   +22     
=======================================
+ Hits        59169    59185   +16     
- Misses     132956   132980   +24     

Flag	Coverage Δ
opencti	30.79% <ø> (+<0.01%)	:arrow_up:
opencti-front	2.46% <ø> (+0.01%)	:arrow_up:
opencti-graphql	68.19% <ø> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[aHenryJard]

I do not understand why there are lot of count changes while no test has been added?

It's because the oasis-test that is importing half data is now run before resolvers when previously it was run after

(and also we should not just count data but verify what a test expect to see)

I have no choice because this part of data is required to run rule tests

[SarahBocognano]

Why the folders are named like :

• 02-dataInjection
• 03-integration
• 10-stream
• 11-sync
• 20-rules ... Why such a gap in numbers ? why not be : 04-stream, 05-sync etc...

[aHenryJard]

Why the folders are named like :

• 02-dataInjection
• 03-integration
• 10-stream
• 11-sync
• 20-rules ... Why such a gap in numbers ? why not be : 04-stream, 05-sync etc...

To allow insert new folder names in the middle easily, when we need and keep deprecated one always the latest

[MTorbay-Filigran]

All the commands are working for me except for this one test:ci-rules-and-others, that give me an error :

[aHenryJard]

All the commands are working for me except for this one test:ci-rules-and-others, that give me an error :

It's working on CI for example here https://github.com/OpenCTI-Platform/opencti/actions/runs/20266501213/job/58191344965

Do you have the test worker up and running along with that yarn test:ci-rules-and-others ?