How can I visit the fdb server from outside of Kubernetes , eg, my desktop?

[xhuaustc]

It seems that the operator only supports ClusterIP as the public ip. But how can I visit the fdb server from outside of Kubernetes, eg, my desktop?

[johscheuer]

Hello, could you add some additional context e.g. what are you trying? Just to give you a better answer. Depending on what you want to achieve you can either use kubectl exec and jump into one of the FoundationDB Pods and use fdbcli there. Otherwise you have to deploy you application inside the Kubernetes cluster you can look at the data-loader example or you could try kubefwd. I never used kubefwd with FoundationDB so it might work or not.

[johscheuer]

If you want to add support for different Service types that should be relatively easy by adding the required field in the cluster spec and then update the GetService method: https://github.com/FoundationDB/fdb-kubernetes-operator/blob/v0.37.0/controllers/pod_models.go#L80-L104

[xhuaustc]

@johscheuer Thanks for you replies. My env as below. cluster confiuration define file

apiVersion: apps.foundationdb.org/v1beta1
kind: FoundationDBCluster
metadata:
  name: sample-cluster
  namespace: fdb
spec:
  processCounts:
    log: 1
    storage: 1
    stateless: 1
  processes:
    general:
      volumeClaimTemplate:
        spec:
          resources:
            requests:
              storage: 40Gi
          storageClassName: local-storage
  databaseConfiguration:
    redundancy_mode: single
  services:
    publicIPSource: service
  version: 6.3.15

after deploy, fdb pod created.

panxh:~/Desktop/test panxh$ kubectl get pod
NAME                               READY     STATUS    RESTARTS   AGE
fdb-fdb-operator-5f6bc7898-xxc2f   1/1       Running   0          3d
sample-cluster-log-1               2/2       Running   0          1h
sample-cluster-stateless-1         2/2       Running   0          1h
sample-cluster-storage-1           2/2       Running   0          49m

panxh:~/Desktop/test panxh$ kubectl exec sample-cluster-storage-1 -- fdbcli --exec  status
Defaulted container "foundationdb" out of: foundationdb, foundationdb-kubernetes-sidecar, foundationdb-kubernetes-init (init)
Using cluster file `/var/dynamic-conf/fdb.cluster'.

Configuration:
  Redundancy mode        - single
  Storage engine         - ssd-2
  Coordinators           - 1
  Desired Proxies        - 3
  Desired Resolvers      - 1
  Desired Logs           - 3
  Desired Remote Logs    - -1
  Desired Log Routers    - -1
  Usable Regions         - 1

Cluster:
  FoundationDB processes - 3 (less 0 excluded; 3 with errors)
  Zones                  - 2
  Machines               - 2
  Memory availability    - 8.0 GB per process on machine with least available
  Fault Tolerance        - 0 machines
  Server time            - 06/28/21 08:51:58

Data:
  Replication health     - Healthy
  Moving data            - 0.000 GB
  Sum of key-value sizes - 0 MB
  Disk space used        - 210 MB

Operating space:
  Storage server         - 94.3 GB free on most full server
  Log server             - 47.1 GB free on most full server

Workload:
  Read rate              - 7 Hz
  Write rate             - 0 Hz
  Transactions started   - 3 Hz
  Transactions committed - 0 Hz
  Conflict rate          - 0 Hz

Backup and DR:
  Running backups        - 0
  Running DRs            - 0

Client time: 06/28/21 08:52:47

I expot the svc with kubectl port-forward svc/sample-cluster-storage-1 14501:4501, and try to connect with fdbcli with the cluster config context sample_cluster:[email protected]:14501.

panxh:~/Desktop/test panxh$ fdbcli -C a.cluster
Using cluster file `a.cluster'.

The database is unavailable; type `status' for more information.

Welcome to the fdbcli. For help, type `help'.
fdb>

I tried some other ways, eg kubefwd, nodeport .. but all failed.

[xhuaustc]

@johscheuer My app which accesses to fdb is under develping, so it runs in my laptop. And the K8s cluster's network driver is calico.

[brownleej]

Every FDB process has a public address (IP and port), and everything that connects to that process must be able to connect on that public address. Because of that, I don't think a port-forwarding config will work. We have an issue tracking more flexible service configurations (#391) which may cover your use case.

[mengzhbj]

If you want to add support for different Service types that should be relatively easy by adding the required field in the cluster spec and then update the GetService method: https://github.com/FoundationDB/fdb-kubernetes-operator/blob/v0.37.0/controllers/pod_models.go#L80-L104

@johscheuer Hi Dear Expert, the fdb operator you and other guys created help our project a lot. Appreciate! Continue with the discussion in https://forums.foundationdb.org/t/how-to-connect-to-foundationdb-which-is-setup-on-an-openshift-cluster-from-outside-of-the-cluster/3035/4, I want to add support for LoadBalancer. @brownleej has an issue #391 seems covering this topic but no update now. Some things I'm not clearly enough, could you please help confirm? Thanks in advance.

1. Why build service only for storage process? To make the fdb cluster accessible from outside, I think we need to build services for all the storage processes, proxy processes, and cluster controller process.
2. Do I have to add support for NodePort before supporting LoadBalancer? Since LoadBalancer is addition to NodePort type.
3. I haven't debug but if simply add return like Type: corev1.ServiceTypeClusterIP to Type: corev1.ServiceTypeNodePort and Type: corev1.ServiceTypeLoadBalancer could make the service returned conform to my intent service type? What else part could be affected by bringing in the new service type?

[johscheuer]

If you want to add support for different Service types that should be relatively easy by adding the required field in the cluster spec and then update the GetService method: https://github.com/FoundationDB/fdb-kubernetes-operator/blob/v0.37.0/controllers/pod_models.go#L80-L104

@johscheuer Hi Dear Expert, the fdb operator you and other guys created help our project a lot. Appreciate! Continue with the discussion in https://forums.foundationdb.org/t/how-to-connect-to-foundationdb-which-is-setup-on-an-openshift-cluster-from-outside-of-the-cluster/3035/4, I want to add support for LoadBalancer. @brownleej has an issue #391 seems covering this topic but no update now. Some things I'm not clearly enough, could you please help confirm? Thanks in advance.

1. Why build service only for storage process? To make the fdb cluster accessible from outside, I think we need to build services for all the storage processes, proxy processes, and cluster controller process.

I'm not sure to what you refer but I never said that the service should be only used for storage Pods and I think in fact this is currently not support and we can only enable services for the whole cluster.

2. Do I have to add support for NodePort before supporting LoadBalancer? Since LoadBalancer is addition to NodePort type.

I don't think that's required and we want to have a generic way to define a service type, currently it's hardcoded but a user should be able to select any service type (the default should be clusterIP).

3. I haven't debug but if simply add return like Type: corev1.ServiceTypeClusterIP to Type: corev1.ServiceTypeNodePort and Type: corev1.ServiceTypeLoadBalancer could make the service returned conform to my intent service type? What else part could be affected by bringing in the new service type?

I think that is actually everything that you need to do. In order to have this merged we want to have a setting under the RoutingConfig to let people define the service type. That should be the only code part that is affect and should work fine, there might be some details how we have to handle the public address based on the different service types but that would be something you can explore by testing the different service types.

[mengzhbj]

@johscheuer Hello, I've implemented something, do you think it's valuable to go back to community?

1. Support IBM Public Cloud Gen2 LoadBalancer Service type.
2. Service generate in the same zone with its relevant pod.
3. Create service only for storage/proxy/cluster_controller processes which are required be visited from outside. Any comments if we want to contribute it?

[johscheuer]

I think that would be valuable for the community and would fit under: https://github.com/FoundationDB/fdb-kubernetes-operator/issues/391 as long as the implementation is generic and allows users to define the service type.