= Hybrid Cloud :experimental: :cloud-1: gcp :cloud-1-weight: 0 :cloud-2: azr :cloud-2-weight: 15 :cloud-3: aws :cloud-3-weight: 20

image:https://img.shields.io/badge/OpenShift-v4.5.x-red?style=for-the-badge[link=https://try.openshift.com] image:https://img.shields.io/badge/OpenShift%20Serverless-v1.10-red?style=for-the-badge[link= https://www.openshift.com/learn/topics/serverless] image:https://img.shields.io/badge/skupper-v0.3.0-red?style=for-the-badge[link= https://skupper.io]

== Download Sources

Lets use the following variables to refer to the source repositories :

• $HYBRID_CLOUD_HOME -- https://github.com/redhat-developer-demos/hybrid-cloud-serverless
• $BACKEND_APP_HOME -- https://github.com/redhat-developer-demos/hybrid-cloud-app-backend
• $FRONTEND_APP_HOME -- https://github.com/redhat-developer-demos/hybrid-cloud-app-frontend

[source,bash]

git clone https://github.com/redhat-developer-demos/hybrid-cloud-serverless export HYBRID_CLOUD_HOME=pwd/hybrid-cloud-serverless git clone https://github.com/redhat-developer-demos/hybrid-cloud-app-backend export BACKEND_APP_HOME=pwd/hybrid-cloud-app-backend git clone https://github.com/redhat-developer-demos/hybrid-cloud-app-frontend export FRONTEND_APP_HOME=pwd/hybrid-cloud-app-frontend

== Prerequisites

• https://try.openshift.com[OpenShift4] cluster on three clouds with following components installed: ** https://www.openshift.com/learn/topics/serverless[OpenShift Serverless]

• https://skupper.io/releases/index.html[skupper]

• https://mikefarah.gitbook.io/yq/[yq]

Let us call the OpenShift Clusters as Cloud-1({cloud-1}), Cloud-2({cloud-2}) and Cloud-3({cloud-3}).

The following table shows the cloud and what components gets installed on each of them:

.Clouds and Components [cols="<2,^1,^1,^1", options="header"] |=== | Component | Cloud-1 | Cloud-2 | Cloud-3 | Cloud Provider | {cloud-1} | {cloud-2} | {cloud-3} | Backend | ✓ | ✓ | ✓ | Frontend | ✓ | ❌ | ❌ | Generate Site Token(token.yaml) | ✓ | ❌ | ❌ | Weight | 0 | 15 | 20 |===

[NOTE]

• You can use any cloud provider for any OpenShift4 supported cloud, the gcp/aws/azr is used in this example.
• Weight controls how many requests that cloud can handle before skupper bursts them out to other clouds ====

Before the backend or frontend applications are deployed, execute the following on each cloud:

[source,bash]

cd $HYBRID_CLOUD_HOME oc apply -k k8s/core oc project hybrid-cloud-demo oc apply -k k8s/rbac

=== Cloud-1

=== Add Image Streams

[source,bash]

oc import-image hybrid-cloud-backend --confirm --all --from quay.io/rhdevelopers/hybrid-cloud-demo-backend

set local lookup policy

oc set image-lookup hybrid-cloud-backend oc import-image hybrid-cloud-frontend --confirm --all --from quay.io/rhdevelopers/hybrid-cloud-demo-frontend

set local lookup policy

oc set image-lookup hybrid-cloud-frontend

[source,bash,subs="macros+,attributes+"]

oc apply -k k8s/skupper/{cloud-1}

=== Cloud-2

=== Add Image Streams

[source,bash,subs="macros+,attributes+"]

oc import-image hybrid-cloud-backend --confirm --all --from quay.io/rhdevelopers/hybrid-cloud-demo-backend

set local lookup policy

oc set image-lookup hybrid-cloud-backend

[source,bash,subs="macros+,attributes+"]

oc apply -k k8s/skupper/{cloud-2}

=== Cloud-3

=== Add Image Streams

[source,bash,subs="macros+,attributes+"]

oc import-image hybrid-cloud-backend --confirm --all --from quay.io/rhdevelopers/hybrid-cloud-demo-backend

set local lookup policy

oc set image-lookup hybrid-cloud-backend

[source,bash,subs="macros+,attributes+"]

oc apply -k k8s/skupper/{cloud-3}

Run the following commands on Cloud-1, Cloud-2 and Cloud-3 to wait for skupper deployments to be ready:

[source,bash,subs="macros+,attributes+"]

oc rollout status deployment skupper-site-controller oc rollout status deployment skupper-router oc rollout status deployment skupper-service-controller

== Backend Deployment

On all clouds Cloud-1, Cloud-2 and Cloud-3, run the following commands to deploy Knative backend:

Deploy backend application(Knative):

[source,bash,subs="macros+,attributes+"]

cd $BACKEND_APP_HOME oc apply -k k8s/knative

Run the following command to check the status:

[source,bash,subs="macros+,attributes+"]

oc get pods,svc,ksvc

A successful deployments of core,rbac,skupper and backend components, should show an output like:

[source,text]

NAME READY STATUS RESTARTS AGE pod/hybrid-cloud-backend-p948k-deployment-b49c9569b-ggv8z 2/2 Running 0 26s pod/skupper-router-56c4544bbc-dhckt 3/3 Running 0 43m pod/skupper-service-controller-5bcf486799-v2hl2 2/2 Running 0 43m pod/skupper-site-controller-5cf967f858-z2dx8 1/1 Running 0 43m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/hybrid-cloud-backend ExternalName kourier-internal.knative-serving-ingress.svc.cluster.local 21s service/hybrid-cloud-backend-p948k ClusterIP 172.30.223.229 80/TCP 26s service/hybrid-cloud-backend-p948k-private ClusterIP 172.30.140.107 80/TCP,9090/TCP,9091/TCP,8022/TCP 26s service/hybrid-cloud-backend-skupper LoadBalancer 172.30.1.23 80:31554/TCP 29s service/skupper-controller ClusterIP 172.30.119.15 443/TCP 43m service/skupper-internal ClusterIP 172.30.205.136 55671/TCP,45671/TCP 43m service/skupper-messaging ClusterIP 172.30.14.214 5671/TCP 43m service/skupper-router-console ClusterIP 172.30.72.116 443/TCP 43m

NAME URL LATESTCREATED LATESTREADY READY REASON service.serving.knative.dev/hybrid-cloud-backend http://hybrid-cloud-backend.hybrid-cloud-demo.svc.cluster.local hybrid-cloud-backend-p948k hybrid-cloud-backend-p948k True

== Connecting Clouds

On Cloud-1, run the following command to create site-token secret:

[source,bash]

cd $HYBRID_CLOUD_HOME oc apply -k k8s/van

The site-token seceret will be used to connect clouds Cloud-2 and Cloud-3 to Cloud-1 forming a Virtual Application Network(VAN).

Run the following command to export the site-token secret:

[source,bash]

oc get secret -n hybrid-cloud-demo site-token -o yaml > $HYBRID_CLOUD_HOME/token.yaml

To connect Cloud-2 and Cloud-3 with Cloud-1, run the following command on Cloud-2 and Cloud-3:

=== Cloud-2

Make sure the Cloud-2 starts to spill over after 4 requests in queue:

[source,bash,subs="macros+,attributes+"]

yq w $HYBRID_CLOUD_HOME/token.yaml 'metadata.annotations[skupper.io/cost]' --tag '!!str' {cloud-2-weight} | oc create -n hybrid-cloud-demo -f -

=== Cloud-3

Make sure the Cloud-3 starts to spill over after 2 requests in queue:

[source,bash,subs="macros+,attributes+"]

yq w $HYBRID_CLOUD_HOME/token.yaml 'metadata.annotations[skupper.io/cost]' --tag '!!str' {cloud-3-weight} | oc create -n hybrid-cloud-demo -f -

A successfull connection can be verified using the following commands, on Cloud-1:

[source,bash,subs="macros+,attributes+"]

export SKUPPER_POD=$(oc get pods --selector=application=skupper-router -ojsonpath='{.items[0].metadata.name}') kubectl exec -it $SKUPPER_POD -- qdmanage query --type node

The command should show an output as shown below:

[source,json,subs="macros+,attributes+"]

[ { "index": 0, "nextHop": "(self)", "name": "router.node/hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd", "validOrigins": [], "linkState": [ "hybrid-cloud-azr-skupper-router-675cb597fc-75g2x", "hybrid-cloud-aws-skupper-router-5946d4544f-mgswf" ], "instance": 1599839762, "identity": "router.node/hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd", "protocolVersion": 1, "lastTopoChange": 1599840055, "type": "org.apache.qpid.dispatch.router.node", "id": "hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd", "address": "amqp:/_topo/0/hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd" }, { "routerLink": 0, "name": "router.node/hybrid-cloud-aws-skupper-router-5946d4544f-mgswf", "index": 1, "validOrigins": [ "hybrid-cloud-azr-skupper-router-675cb597fc-75g2x" ], "protocolVersion": 1, "linkState": [ "hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd" ], "instance": 1599840049, "cost": {cloud-2-weight}, "address": "amqp:/_topo/0/hybrid-cloud-aws-skupper-router-5946d4544f-mgswf", "type": "org.apache.qpid.dispatch.router.node", "id": "hybrid-cloud-aws-skupper-router-5946d4544f-mgswf", "identity": "router.node/hybrid-cloud-aws-skupper-router-5946d4544f-mgswf" }, { "routerLink": 1, "name": "router.node/hybrid-cloud-azr-skupper-router-675cb597fc-75g2x", "index": 2, "validOrigins": [ "hybrid-cloud-aws-skupper-router-5946d4544f-mgswf" ], "protocolVersion": 1, "linkState": [ "hybrid-cloud-gcp-skupper-router-7b9fcf9575-knrsd" ], "instance": 1599840051, "cost": {cloud-3-weight}, "address": "amqp:/_topo/0/hybrid-cloud-azr-skupper-router-675cb597fc-75g2x", "type": "org.apache.qpid.dispatch.router.node", "id": "hybrid-cloud-azr-skupper-router-675cb597fc-75g2x", "identity": "router.node/hybrid-cloud-azr-skupper-router-675cb597fc-75g2x" } ]

== Verify Status

=== Cloud-1

Running skupper status on the clouds should show the following output:

[source,text]

Skupper is enabled for namespace '"hybrid-cloud-demo" in interior mode'. It is connected to 2 other sites. It has 1 exposed service.

=== Cloud-2

Running skupper status on the clouds should show the following output:

[source,text]

Skupper is enabled for namespace '"hybrid-cloud-demo" in interior mode'. It is connected to 2 other sites (1 indirectly). It has 1 exposed service.

=== Cloud-3

Running skupper status on the clouds should show the following output:

[source,text]

Skupper is enabled for namespace '"hybrid-cloud-demo" in interior mode'. It is connected to 2 other sites (1 indirectly). It has 1 exposed service.

[NOTE]

Since Cloud-1 is where we generated the site-token secret to connect to other sites, makes it directly connected to other clouds Cloud-2 and Cloud-3. For other clouds you will see one direct to Cloud-1 and one indirect connection to other cloud.

== Verify Exposed Services

You can verify that, in all connected clouds running the following command skupper list-exposed shows the following output:

When Knative backend was deployed:

[source,bash]

Services exposed through Skupper: hybrid-cloud-backend-skupper (http port 80) with targets => hybrid-cloud-backend.hybrid-cloud-demo name=hybrid-cloud-backend.hybrid-cloud-demo

When standard Kubernetes (vanilla) backend was deployed:

[source,bash]

Services exposed through Skupper: hybrid-cloud-backend (http port 80) with targets => hybrid-cloud-backend.hybrid-cloud-demo name=hybrid-cloud-backend.hybrid-cloud-demo

== Frontend Deployment

On Cloud-1 deploy the frontend by running the following command:

[source,bash]

cd $FRONTEND_APP_HOME oc apply -k k8s/knative

Get the URL to access the frontend application:

[source,bash]

export API_URL=http://$(oc get route -n hybrid-cloud-demo hybrid-cloud-frontend -ojsonpath='{.spec.host}')

== Burst Testing

It is possible to verify the brust without user input using the following https://github.com/rakyll/hey[hey] scripts:

=== Cloud-1 burst to Cloud-2

In order to burst from Cloud-1 to Cloud-2, you need to send atleast {cloud-2-weight} requests to the API:

[source,bash,subs="macros+,attributes+"]

hey -z 2s -c 20 -m POST -d '{"text": "1+2","uppercase": false,"reverse": false}' -H "Content-Type: application/json" $API_URL/api/send-request

=== Cloud-1 burst to Cloud-2 burst to Cloud-3

In order to burst from Cloud-1 to Cloud-2, you need to send atleast {cloud-2-weight} + {cloud-3-weight} = 35 requests to the API:

[source,bash,subs="macros+,attributes+"]

hey -z 2s -c 35 -m POST -d '{"text": "1+2+3","uppercase": false,"reverse": false}' -H "Content-Type: application/json" $API_URL/api/send-request