redhat-developer
Validate test script against a ocp+proxy setup environment
/kind user-story /area testing
User Story
As a user I want to run odo test script inside a cluster which has proxy setup
Acceptance Criteria
- [ ] odo test should run successfully under such scenario
Scope Of Issue:
Verify the steps manually
- [ ] Create a flexy job (ocp+proxy)
- [ ] Configure the cluster using the make target (make configure-installer....)
- [ ] Run the test script
Links
- Feature Request: NA
- Related issue: NA
/kind user-story
Hi @prietyc123 , We did some validation on disconnected env before. About the proxy environment, Is there any difference or special requirements ? Thanks.
Hi @prietyc123 & @zhengxiaomei123
Following are some of the changes implemented in ODO to make it work for PowerVS cluster (Proxy enabled cluster):
PowerVS is proxy based cluster.
Initially, we used to set the HTTP_PROXY and HTTPS_PROXY variables with odo create command.
- helper.CmdShouldPass("odo", "create", "--s2i", cmpType, srcType+"-app", "--project", project, "--context", context, "--app", appName)
+ helper.CmdShouldPass("odo", "create", "--s2i", cmpType, srcType+"-app", "--project", project, "--context", context, "--app", appName, "--env", "HTTP_PROXY=<http_proxy_value>", "--env", "HTTPS_PROXY=<http_proxy_value>")
We observed that, odo doesn't allow --env flag with --starter. Hence starter projects failed to execute.
We added changes, to run odo config set after odo create.
With this approach, most of the things works. But odo create … --now will fail.
To take care of proxy environment, thought of implementing code changes inside odo. There are 2 paths of odo push:
- With --starter (devfile): thought of pushing HTTP_PROXY and HTTPS_PROXY values to containers created by ODO.
diff --git a/pkg/devfile/adapters/kubernetes/component/adapter.go b/pkg/devfile/adapters/kubernetes/component/adapter.go
index b3b195e70..d5864abb3 100644
--- a/pkg/devfile/adapters/kubernetes/component/adapter.go
+++ b/pkg/devfile/adapters/kubernetes/component/adapter.go
@@ -284,6 +284,9 @@ func (a Adapter) createOrUpdateComponent(componentExists bool, ei envinfo.EnvSpe
return fmt.Errorf("No valid components found in the devfile")
}
+ // Add Enviroment variables if available
+ utils.UpdateProxyVariablesIfAvailable(&containers)
+
// Add the project volume before generating init containers
utils.AddOdoProjectVolume(&containers)
diff --git a/pkg/devfile/adapters/kubernetes/utils/utils.go b/pkg/devfile/adapters/kubernetes/utils/utils.go
index a33bcbceb..7d535e472 100644
--- a/pkg/devfile/adapters/kubernetes/utils/utils.go
+++ b/pkg/devfile/adapters/kubernetes/utils/utils.go
@@ -2,6 +2,8 @@ package utils
import (
"fmt"
+ "net/url"
+ "os"
"strconv"
"strings"
@@ -81,6 +83,62 @@ func isEnvPresent(EnvVars []corev1.EnvVar, envVarName string) bool {
return isPresent
}
+func UpdateProxyVariablesIfAvailable(containers *[]corev1.Container) {
+ httpProxy := os.Getenv("Http_Proxy")
+ httpsProxy := os.Getenv("Https_Proxy")
+ if httpProxy == "<no value>" || httpsProxy == "<no value>" || httpProxy == "" || httpsProxy == "" {
+ return
+ }
+
+ httpProxyURL, err := url.Parse(httpProxy)
+ if err != nil {
+ return
+ }
+
+ httpsProxyURL, err := url.Parse(httpsProxy)
+ if err != nil {
+ return
+ }
+
+ mavenProxyOptions := fmt.Sprintf("-DproxySet=true -Dhttp.proxyHost=%s -Dhttp.proxyPort=%s -Dhttps.proxyHost=%s -Dhttps.proxyPort=%s",
+ httpProxyURL.Hostname(), httpProxyURL.Port(), httpsProxyURL.Hostname(), httpsProxyURL.Port())
+
+ for i, container := range *containers {
+ isHTTPProxyEnvAvailable := false
+ isHTTPSProxyEnvAvailable := false
+ for _, env := range container.Env {
+ if env.Name == "HTTP_PROXY" {
+ isHTTPProxyEnvAvailable = true
+ }
+ if env.Name == "HTTPS_PROXY" {
+ isHTTPSProxyEnvAvailable = true
+ }
+ }
+ if !isHTTPProxyEnvAvailable {
+ fmt.Println("[KP-DBG] Adding HTTP_PROXY as ", httpProxy)
+ container.Env = append(container.Env, corev1.EnvVar{
+ Name: "HTTP_PROXY",
+ Value: httpProxy,
+ })
+ }
+ if !isHTTPSProxyEnvAvailable {
+ fmt.Println("[KP-DBG] Adding HTTPS_PROXY as ", httpsProxy)
+ container.Env = append(container.Env, corev1.EnvVar{
+ Name: "HTTPS_PROXY",
+ Value: httpsProxy,
+ })
+ }
+
+ fmt.Println("[KP-DBG] Adding MAVEN_OPTS as ", mavenProxyOptions)
+ container.Env = append(container.Env, corev1.EnvVar{
+ Name: "MAVEN_OPTS",
+ Value: mavenProxyOptions,
+ })
+
+ (*containers)[i] = container
+ }
+}
+
// AddOdoProjectVolume adds the odo project volume to the containers
func AddOdoProjectVolume(containers *[]corev1.Container) {
for i, container := range *containers {
- With --s2i : Adding Proxy environment variables to config.yaml file.
diff --git a/pkg/config/config.go b/pkg/config/config.go
index 651bc56f4..dbfe9ada3 100644
--- a/pkg/config/config.go
+++ b/pkg/config/config.go
@@ -6,6 +6,7 @@ import (
"net/url"
"os"
"path/filepath"
+ "sort"
"strconv"
"strings"
@@ -361,6 +362,9 @@ func (lci *LocalConfigInfo) GetComponentSettings() ComponentSettings {
// SetComponentSettings sets the componentSettings from to the local config and writes to the file
func (lci *LocalConfigInfo) SetComponentSettings(cs ComponentSettings) error {
+ proxyEnvs := getProxyDetailsIfAvailable()
+ cs.Envs = proxyEnvs.Merge(cs.Envs)
+ // Sorting is required to keep env variables in an order. To make test case to pass.
+ sort.Slice(cs.Envs, func(i, j int) bool { return cs.Envs[i].Name < cs.Envs[j].Name })
lci.componentSettings = cs
return lci.writeToFile()
}
@@ -470,6 +474,33 @@ func (lc *LocalConfig) GetMaxCPU() string {
return util.GetStringOrEmpty(lc.componentSettings.MaxCPU)
}
+func getProxyDetailsIfAvailable() (envList EnvVarList) {
+ httpProxy := os.Getenv("Http_Proxy")
+ httpsProxy := os.Getenv("Https_Proxy")
+ if httpProxy == "<no value>" || httpsProxy == "<no value>" || httpProxy == "" || httpsProxy == "" {
+ return
+ }
+
+ httpProxyURL, err := url.Parse(httpProxy)
+ if err != nil {
+ return
+ }
+
+ httpsProxyURL, err := url.Parse(httpsProxy)
+ if err != nil {
+ return
+ }
+
+ mavenProxyOptions := fmt.Sprintf("-DproxySet=true -Dhttp.proxyHost=%s -Dhttp.proxyPort=%s -Dhttps.proxyHost=%s -Dhttps.proxyPort=%s",
+ httpProxyURL.Hostname(), httpProxyURL.Port(), httpsProxyURL.Hostname(), httpsProxyURL.Port())
+
+ envList = append(envList, EnvVar{Name: "HTTP_PROXY", Value: httpProxy})
+ envList = append(envList, EnvVar{Name: "HTTPS_PROXY", Value: httpsProxy})
+ envList = append(envList, EnvVar{Name: "MAVEN_OPTS", Value: mavenProxyOptions})
+
+ return envList
+}
+
// GetEnvs returns the Envs, returns empty if nil
func (lc *LocalConfig) GetEnvs() EnvVarList {
if lc.componentSettings.Envs == nil {
Notes:
- Adding MAVEN_OPTS to make sure, there won't be any issues with JAVA projects compiled with maven.
- Code cleanup is very much required.
With the above changes, all tests are working fine.
Second part of this scenario is getting proxy variables:
Initially tried of getting proxy details inside the odo code using below code:
type Proxy struct {
HttpProxy string `json:"httpProxy"`
HttpsProxy string `json:"httpsProxy"`
NoProxy string `json:"noProxy"`
}
proxyDetails := Proxy{}
coreGet := c.kubeClient.KubeClient.CoreV1().RESTClient().Get()
rawProxyValues, err := coreGet.AbsPath("/apis/config.openshift.io/v1/proxies").Do().Raw()
if err != nil {
// when using Minishift (or plain 'oc cluster up' for that matter) with OKD 3.11, the version endpoint is missing...
klog.V(3).Infof("Unable to get OpenShift Version - endpoint '/apis/config.openshift.io/v1/proxies' doesn't exist")
return err
}
proxyJSON := map[string]*json.RawMessage{}
err = json.Unmarshal([]byte(rawProxyValues), &proxyJSON)
if err != nil {
klog.V(3).Infof("Failed to parse json data")
return err
}
proxyItems := []map[string]*json.RawMessage{}
err = json.Unmarshal(*(proxyJSON["items"]), &proxyItems)
if err != nil {
klog.V(3).Infof("Failed to parse Proxy items")
return err
}
for _, proxy := range proxyItems {
if string(*proxy["kind"]) == "\"Proxy\"" {
if err = json.Unmarshal(*proxy["status"], &proxyDetails); err != nil {
klog.V(3).Infof("Failed to parse status of the proxy")
return err
}
break
}
}
Issue with this approach is REST call to /apis/config.openshift.io/v1/proxies can be done by admin users. Getting error with developer user. All the tests run as developers, so not able to get the proxy details via odo code. To fix this added following lines into scripts/openshiftci-presubmit-all-test.sh
export Http_Proxy=$(oc get proxy -o go-template='{{range .items}}{{.spec.httpProxy}}{{"\n"}}{{end}}')
export Https_Proxy=$(oc get proxy -o go-template='{{range .items}}{{.spec.httpsProxy}}{{"\n"}}{{end}}')
Please let me know your views on the above changes.
Hi @prietyc123 @dharmit any update on this issue ? Please let me know do you have any plans on this issue.
Hi @prietyc123 @dharmit any update on this issue ? Please let me know do you have any plans on this issue.
@dharmit @girishramnani @prietyc123 any update on this ?
@kandarpamalipeddi at the moment, this isn't a priority. Is this blocking you folks in any way?
@prietyc123 when do you plan to pick this up from the QE side?
@dharmit, @prietyc123
We are not blocked with this. Currently our tests are working fine with the work around I mentioned in the Comment.
Do you guys have any plans to implement code changes to make odo work on Proxy based cluster ? Or this story is to track the testing status of the Odo on proxy based cluster ?
Do you guys have any plans to implement code changes to make odo work on Proxy based cluster ? Or this story is to track the testing status of the Odo on proxy based cluster ?
@kandarpamalipeddi From testing end this is not the priority currently. Before this issue I guess we need to focus on https://github.com/openshift/odo/issues/4409 as part of QE plan. But again its of low priority currently. As you said work arounds going well for you, sounds like its not a blocker from your end too. As far as code change is concerned, I am pretty sure that's also not a priority from Dev end. However I would like someone from Dev to chime in and update the status. @dharmit @kadel @feloy WDYT about comment
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
