[WIP] feat(serving): light weighted traffic control for inference

[ccchenjiahuan]

Ⅰ. Describe what this PR does

Functions currently implemented:

• [x] istio-less ingress gateway
• [x] kubedl serving reconcile logic modification

I have done some tests on the ingress gateway part, but haven't got a chance to test the changes with kubedl, I think we can review the gateway part first.

Functions currently known but not yet implemented:

• [ ] The ingress gateway watches all ingresses currently, it needs to be configured to watch specific inference cr related ingresses
• [ ] Create a specific service account for the ingress gateway pod in the serving reconciliation

II. Does this pull request fix one issue?

resolves https://github.com/kubedl-io/kubedl/issues/160

[codecov-commenter]

Codecov Report

Merging #184 (1bdb54e) into master (f408cc8) will increase coverage by 0.88%. The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #184      +/-   ##
==========================================
+ Coverage   21.74%   22.62%   +0.88%     
==========================================
  Files          73       75       +2     
  Lines        4374     4557     +183     
==========================================
+ Hits          951     1031      +80     
- Misses       3292     3392     +100     
- Partials      131      134       +3     

Flag	Coverage Δ
unittests	22.62% <0.00%> (+0.88%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
controllers/serving/inference_controller.go	0.00% <0.00%> (ø)
controllers/serving/utils.go	0.00% <0.00%> (ø)
controllers/tensorflow/tfjob_controller.go	44.77% <0.00%> (-2.24%)	:arrow_down:
pkg/job_controller/job.go	13.61% <0.00%> (ø)
controllers/mars/marsjob_controller.go	0.00% <0.00%> (ø)
controllers/xgboost/xgboostjob_controller.go	0.00% <0.00%> (ø)
pkg/gang_schedule/batch_scheduler/scheduler.go	68.88% <0.00%> (ø)
pkg/gang_schedule/volcano_scheduler/scheduler.go	68.88% <0.00%> (ø)
pkg/gang_schedule/coscheduler/scheduler.go	64.70% <0.00%> (+61.58%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f408cc8...1bdb54e. Read the comment docs.

[ccchenjiahuan]

/cc @SimonCqk @jian-he