ciao icon indicating copy to clipboard operation
ciao copied to clipboard

Published 20 hours ago • verified publisher icon caicloud

Generalized namespaces, added limits and other workers

Open ashahab opened this issue 7 years ago • 8 comments

What this PR does / why we need it:

Add your description

Which issue(s) this PR is related to (optional, link to 3rd issue(s)):

Fixes #

Reference to #

Special notes for your reviewer:

/cc @your-reviewer

Release note:

NONE

ashahab avatar Oct 08 '18 03:10 ashahab

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To fully approve this pull request, please assign additional approvers. We suggest the following additional approver: ddysher

Assign the PR to them by writing /assign @ddysher in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

caicloud-bot avatar Oct 08 '18 03:10 caicloud-bot

Travis tests have failed

Hey @ashahab, Please read the following log in order to understand the failure reason. It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/s2i/img/img.go:1::warning: file is not goimported (goimports)
pkg/s2i/simple/simple.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:88:21:warning: error return value not checked (m.S2IClient.Cleanup(parameter)) (errcheck)
pkg/backend/kubeflow/generator/cm.go:53:2:warning: should merge variable declaration with assignment on next line (S1021) (gosimple)
pkg/interpreter/simple/simple.go:53:12:warning: unnecessary conversion (unconvert)
cmd/kubeflow-kernel/command/run.go:1::warning: file is not goimported (goimports)
pkg/backend/kubeflow/kubeflow.go:1::warning: file is not goimported (goimports)
pkg/backend/kubeflow/generator/cm.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:1::warning: file is not goimported (goimports)
pkg/s2i/types.go:1::warning: file is not goimported (goimports)
pkg/s2i/configmap/configmap.go:1::warning: file is not goimported (goimports)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/kernel/*.go" 
?   	github.com/caicloud/ciao/pkg/backend	[no test files]
?   	github.com/caicloud/ciao/pkg/backend/kubeflow	[no test files]
=== RUN   TestCMNewTFJob
--- PASS: TestCMNewTFJob (0.00s)
=== RUN   TestCMNewPyTorchJob
--- PASS: TestCMNewPyTorchJob (0.00s)
=== RUN   TestNewTFJob
--- PASS: TestNewTFJob (0.00s)
=== RUN   TestNewPyTorchJob
--- PASS: TestNewPyTorchJob (0.00s)
PASS
coverage: 4.3% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
ok  	github.com/caicloud/ciao/pkg/backend/kubeflow/generator	0.042s
?   	github.com/caicloud/ciao/pkg/config	[no test files]
?   	github.com/caicloud/ciao/pkg/interpreter	[no test files]
anager
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/configmap
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/img
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/mock
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/simple
2018/10/08 03:18:32 Processing line: %framework=tensorflow
2018/10/08 03:18:32 Processing line: %ps=1
2018/10/08 03:18:32 Processing line: %worker=1
2018/10/08 03:18:32 Processing line: some code here.
2018/10/08 03:18:32 Processing line: %framework=tensorflow
2018/10/08 03:18:32 Processing line: %ps=1
2018/10/08 03:18:32 Processing line: some code here.
2018/10/08 03:18:32 Processing line: %framework=tensorflow
2018/10/08 03:18:32 Processing line: %ps=1

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/backend/kubeflow/generator/cm.go:53:2:warning: should merge variable declaration with assignment on next line (S1021) (gosimple)
cmd/kubeflow-kernel/command/run.go:1::warning: file is not goimported (goimports)
pkg/backend/kubeflow/generator/cm.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:1::warning: file is not goimported (goimports)
pkg/s2i/img/img.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:88:21:warning: error return value not checked (m.S2IClient.Cleanup(parameter)) (errcheck)
pkg/backend/kubeflow/kubeflow.go:1::warning: file is not goimported (goimports)
pkg/s2i/types.go:1::warning: file is not goimported (goimports)
pkg/s2i/configmap/configmap.go:1::warning: file is not goimported (goimports)
pkg/s2i/simple/simple.go:1::warning: file is not goimported (goimports)
pkg/interpreter/simple/simple.go:53:12:warning: unnecessary conversion (unconvert)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/kernel/*.go" 
?   	github.com/caicloud/ciao/pkg/backend	[no test files]
?   	github.com/caicloud/ciao/pkg/backend/kubeflow	[no test files]
=== RUN   TestCMNewTFJob
--- PASS: TestCMNewTFJob (0.00s)
=== RUN   TestCMNewPyTorchJob
--- PASS: TestCMNewPyTorchJob (0.00s)
=== RUN   TestNewTFJob
--- PASS: TestNewTFJob (0.00s)
=== RUN   TestNewPyTorchJob
--- PASS: TestNewPyTorchJob (0.00s)
PASS
coverage: 4.3% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
ok  	github.com/caicloud/ciao/pkg/backend/kubeflow/generator	0.041s
?   	github.com/caicloud/ciao/pkg/config	[no test files]
?   	github.com/caicloud/ciao/pkg/interpreter	[no test files]
=== RUN   TestPreprocess
--- FAIL: TestPreprocess (0.00s)
	simple_test.go:70: Expected &{tensorflow 1 1 0   }, got &{ 0 0 0   }
	simple_test.go:70: Expected &{tensorflow 1 0 0   }, got &{ 0 0 0   }
	simple_test.go:70: Expected &{tensorflow 1 0 0   }, got &{ 0 0 0   }
=== RUN   TestGetPreProcessedCode
--- PASS: TestGetPreProcessedCode (0.00s)
FAIL
coverage: 3.1% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
exit status 1
FAIL	github.com/caicloud/ciao/pkg/interpreter/simple	0.036s
exit status 1: warning: no packages being tested depend on github.com/caicloud/ciao/pkg/backend
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/backend/kubeflow
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/backend/kubeflow/generator
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/config
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/interpreter
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/kernel
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/manager
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/configmap
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/img
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/mock
warning: no packages being tested depend on github.com/caicloud/ciao/pkg/s2i/simple
2018/10/08 03:18:35 Processing line: %framework=tensorflow
2018/10/08 03:18:35 Processing line: %ps=1
2018/10/08 03:18:35 Processing line: %worker=1
2018/10/08 03:18:35 Processing line: some code here.
2018/10/08 03:18:35 Processing line: %framework=tensorflow
2018/10/08 03:18:35 Processing line: %ps=1
2018/10/08 03:18:35 Processing line: some code here.
2018/10/08 03:18:35 Processing line: %framework=tensorflow
2018/10/08 03:18:35 Processing line: %ps=1

3rd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/backend/kubeflow/generator/cm.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:1::warning: file is not goimported (goimports)
pkg/s2i/types.go:1::warning: file is not goimported (goimports)
pkg/s2i/configmap/configmap.go:1::warning: file is not goimported (goimports)
pkg/s2i/img/img.go:1::warning: file is not goimported (goimports)
pkg/manager/manager.go:88:21:warning: error return value not checked (m.S2IClient.Cleanup(parameter)) (errcheck)
cmd/kubeflow-kernel/command/run.go:1::warning: file is not goimported (goimports)
pkg/s2i/simple/simple.go:1::warning: file is not goimported (goimports)
pkg/backend/kubeflow/generator/cm.go:53:2:warning: should merge variable declaration with assignment on next line (S1021) (gosimple)
pkg/interpreter/simple/simple.go:53:12:warning: unnecessary conversion (unconvert)
pkg/backend/kubeflow/kubeflow.go:1::warning: file is not goimported (goimports)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/kernel/*.go" 
?   	github.com/caicloud/ciao/pkg/backend	[no test files]
?   	github.com/caicloud/ciao/pkg/backend/kubeflow	[no test files]
=== RUN   TestCMNewTFJob
--- PASS: TestCMNewTFJob (0.00s)
=== RUN   TestCMNewPyTorchJob
--- PASS: TestCMNewPyTorchJob (0.00s)
=== RUN   TestNewTFJob
--- PASS: TestNewTFJob (0.00s)
=== RUN   TestNewPyTorchJob
--- PASS: TestNewPyTorchJob (0.00s)
PASS
coverage: 70.6% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
ok  	github.com/caicloud/ciao/pkg/backend/kubeflow/generator	0.035s	coverage: 70.6% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
?   	github.com/caicloud/ciao/pkg/config	[no test files]
?   	github.com/caicloud/ciao/pkg/interpreter	[no test files]
=== RUN   TestPreprocess
2018/10/08 03:16:58 Processing line: %framework=tensorflow
2018/10/08 03:16:58 Processing line: %ps=1
2018/10/08 03:16:58 Processing line: %worker=1
2018/10/08 03:16:58 Processing line: some code here.
2018/10/08 03:16:58 Processing line: %framework=tensorflow
2018/10/08 03:16:58 Processing line: %ps=1
2018/10/08 03:16:58 Processing line: some code here.
2018/10/08 03:16:58 Processing line: %framework=tensorflow
2018/10/08 03:16:58 Processing line: %ps=1
--- FAIL: TestPreprocess (0.00s)
	simple_test.go:70: Expected &{tensorflow 1 1 0   }, got &{ 0 0 0   }
	simple_test.go:70: Expected &{tensorflow 1 0 0   }, got &{ 0 0 0   }
	simple_test.go:70: Expected &{tensorflow 1 0 0   }, got &{ 0 0 0   }
=== RUN   TestGetPreProcessedCode
--- PASS: TestGetPreProcessedCode (0.00s)
FAIL
coverage: 50.0% of statements in github.com/caicloud/ciao/pkg/backend, github.com/caicloud/ciao/pkg/backend/kubeflow, github.com/caicloud/ciao/pkg/backend/kubeflow/generator, github.com/caicloud/ciao/pkg/config, github.com/caicloud/ciao/pkg/interpreter, github.com/caicloud/ciao/pkg/interpreter/simple, github.com/caicloud/ciao/pkg/kernel, github.com/caicloud/ciao/pkg/manager, github.com/caicloud/ciao/pkg/s2i, github.com/caicloud/ciao/pkg/s2i/configmap, github.com/caicloud/ciao/pkg/s2i/img, github.com/caicloud/ciao/pkg/s2i/mock, github.com/caicloud/ciao/pkg/s2i/simple, github.com/caicloud/ciao/pkg/types
FAIL	github.com/caicloud/ciao/pkg/interpreter/simple	0.002s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/backend
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/backend/kubeflow
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/backend/kubeflow/generator
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/config
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/interpreter
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/kernel
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/manager
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/s2i
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/s2i/configmap
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/s2i/img
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/s2i/mock
warning: no packages being tested depend on matches for pattern github.com/caicloud/ciao/pkg/s2i/simple
TravisBuddy Request Identifier: d8274990-caa8-11e8-bd56-bd6e450aa6f6

TravisBuddy avatar Oct 08 '18 03:10 TravisBuddy

/cc @ddysher

gaocegege avatar Oct 08 '18 06:10 gaocegege

@ashahab: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

caicloud-bot avatar Oct 09 '18 12:10 caicloud-bot

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

/lifecycle stale

caicloud-bot avatar Jan 07 '19 12:01 caicloud-bot

@ashahab Thanks for the PR! I will implement the feature based on your awesome work, since it seems that you have no time to update it.

gaocegege avatar Mar 22 '19 06:03 gaocegege

Actually I do have time next week if you are fine waiting. This is an awesome project, I'd like to contribute.

On Thu, Mar 21, 2019, 11:35 PM Ce Gao [email protected] wrote:

@ashahab https://github.com/ashahab Thanks for the PR! I will implement the feature based on your awesome work, since it seems that you have to time to update it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/caicloud/ciao/pull/52#issuecomment-475509929, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqJPVd17-veS6EVhm4lxpVFjpRED-S2ks5vZHnHgaJpZM4XMMPt .

ashahab avatar Mar 22 '19 15:03 ashahab

@ashahab OK Welcome your contribution! Of course, I can wait for you.

I am a little busy in the past six months while now I think I have time to maintain the project.

gaocegege avatar Mar 25 '19 01:03 gaocegege