[WIP] Implement an Accurate Simulator based on Functional model

[Lemarais]

JIRA: NEMO-478: Implement an Accurate Simulator based on Functional model

Major changes:

• Added run_beam_simulator.sh and Created SimulatorLauncherto run simulator.

  • In addition to the existing arguments, pass hardware specification including network information through -node_json argument

• Created NetworkSimulator and ContainerManageSimulator to manage node and network specification.

  • NetworkSimulator manages network bandwidth and latency between nodes.
  • ContainerManageSimulator manages nodes cpu resources.

• Created PlanSimulator and StreamingPlanSimulator to Emulate on the job level.

  • It manages the size of transferred data between tasks while triggering simulation for ExecutorSimulator *StreamingPlanSimulator has an internal timestamp and iterate cycles until the size of transferred data between tasks reaches equilibrium.

• Created ExecutorSimulator and TaskHarness to Simulate execution of task.

  • TaskHarness has DataFetchSimulator and WriterSimulator.
    • DataFetchSimulator manages the number of tuples that are read from upstream tasks.
    • WriterSimulator manages the number of tuples to transfer to downstream tasks.

Minor changes to note:

Tests for the changes:

• Tested on ubuntu machine

Other comments:

• This is the PR for the GSoC2021 project.

Closes #GITHUB_PR_NUMBER

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[wonook]

Hi @Lemarais , can you share what tasks are left for the completion of this PR?

[Lemarais]

I tried to merge the metric-related PR First. Because until then, I cannot clear up the metric related part of simulator. Then It will be the latest version under the situation that there is no further research. Of course, I think that a review is necessary.