dubbo icon indicating copy to clipboard operation
dubbo copied to clipboard

Published 20 hours ago • verified publisher icon apache

Dubbo3.2.0-beta.5 proxyless mesh模式测试流量路由未成功

Open Silocean opened this issue 1 year ago • 9 comments

  • [ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ask your question here

Environment

dubbo: 3.2.0-beta.5 jdk: 1.8 k8s: 1.25.2 istio: 1.17.1

Steps to reproduce this issue

  1. 按照官方示例（https://cn.dubbo.apache.org/zh-cn/overview/tasks/mesh/proxyless/） 搭建了测试环境。
  2. 使用了官方测试demo（https://github.com/apache/dubbo-samples/tree/master/3-extensions/registry/dubbo-samples-xds） ，修改dubbo版本到3.2.0-beta.5。
  3. 在k8s中部署了两个版本（v1与v2）的provider以及一个consumer。
  4. 使用的路由配置如下所示： image

Expected Behavior

v1版本的pod接收全部consumer流量，v2版本pod没有流量。

Actual Behavior

v1与v2版本流量均有流量，流量路由功能未生效。 image

Silocean avatar Mar 10 '23 15:03 Silocean

请问出现这个问题的原因可能有哪些呢？是dubbo版本问题？或者是路由配置问题？

Silocean avatar Mar 10 '23 15:03 Silocean

目测跟xds router有关，我来看看吧

aamingaa avatar Mar 13 '23 07:03 aamingaa

补充一下provider的报警信息：

2023-03-14T01:21:19.436Z | INFO:  [DUBBO] No value is configured in the registry, the DynamicConfigurationFactory extension[name : xds] does not support as the config center, dubbo version: 1.0-SNAPSHOT, current host: 10.244.0.139
2023-03-14T01:21:19.436Z | Mar 14, 2023 1:21:19 AM org.apache.dubbo.config.deploy.DefaultApplicationDeployer info
2023-03-14T01:21:19.436Z | INFO:  [DUBBO] The registry[<dubbo:registry address="xds://istiod.istio-system.svc:15012" protocol="xds" port="15012" />] will be not used as the config center, dubbo version: 1.0-SNAPSHOT, current host: 10.244.0.139
2023-03-14T01:21:19.638Z | Mar 14, 2023 1:21:19 AM org.apache.dubbo.config.context.ConfigManager info

下面是我的dubbo配置：

dubbo.application.name=dubbo-samples-xds-provider
dubbo.application.metadataServicePort=20885
dubbo.application.metadataServiceProtocol=dubbo
dubbo.registry.address=xds://istiod.istio-system.svc:15012
dubbo.protocol.name=tri
dubbo.protocol.port=50051
dubbo.application.qosEnable=true
dubbo.application.qosAcceptForeignIp=true

看起来像是dubbo并没有把istio控制面识别为注册中心，而且识别出dubbo版本是1.0-SNAPSHOT，但是我pom里配的确实是3.2.0-beta.5，我的配置哪里有问题吗？

Silocean avatar Mar 14 '23 01:03 Silocean

之前测试使用的路由配置host有问题，没有加namespace，所以路由功能没有生效，按照下面的配置修改后又重新做了测试：

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: dubbo-xds-test-vs
  namespace: dubbo-demo
spec:
  hosts:
    - dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
  http:
    - route:
        - destination:
            host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
            subset: v1
            port:
              number: 50051
          weight: 50
        - destination:
            host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
            subset: v2
            port:
              number: 50051
          weight: 50
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: dubbo-xds-test-dr
  namespace: dubbo-demo
spec:
  host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      # Envoy load balancing strategy
      simple: ROUND_ROBIN
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

调用provider直接失败，报错信息如下：

WARNING:  [DUBBO] No provider available after route for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 from registry 10.244.0.252 on the consumer 10.244.0.252 using the dubbo version 1.0-SNAPSHOT. Router snapshot is below: 
[ Parent (Input: 0) (Current Node Output: 0) (Chain Node Output: 0) ] Input: Empty -> Chain Node Output: Empty
  [ MockInvokersSelector (Input: 0) (Current Node Output: 0) (Chain Node Output: 0) Router message: Empty invokers. Directly return. ] Current Node Output: Empty, dubbo version: 1.0-SNAPSHOT, current host: 10.244.0.252, error code: 2-2. This may be caused by No provider available after route for the service, go to https://dubbo.apache.org/faq/2/2 to find instructions. 
Mar 15, 2023 6:59:21 AM org.apache.dubbo.rpc.cluster.directory.AbstractDirectory warn
WARNING:  [DUBBO] No provider available after connectivity filter for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 All validInvokers' size: 0 All routed invokers' size: 0 All invokers' size: 0 from registry istiod.istio-system.svc:15012 on the consumer 10.244.0.252 using the dubbo version 1.0-SNAPSHOT., dubbo version: 1.0-SNAPSHOT, current host: 10.244.0.252, error code: 2-2. This may be caused by provider server or registry center crashed, go to https://dubbo.apache.org/faq/2/2 to find instructions. 
org.apache.dubbo.rpc.RpcException: Failed to invoke the method sayHello in the service org.apache.dubbo.samples.api.GreetingService. No provider available for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 from registry istiod.istio-system.svc:15012 on the consumer 10.244.0.252 using the dubbo version 1.0-SNAPSHOT. Please check if the providers have been started and registered.
 at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.checkInvokers(AbstractClusterInvoker.java:367)
 at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:60)
 at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:341)
 at org.apache.dubbo.rpc.cluster.router.RouterSnapshotFilter.invoke(RouterSnapshotFilter.java:46)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:101)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:52)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.spring.security.filter.ContextHolderParametersSelectedTransferFilter.invoke(ContextHolderParametersSelectedTransferFilter.java:41)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.rpc.cluster.filter.support.ConsumerClassLoaderFilter.invoke(ConsumerClassLoaderFilter.java:40)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.rpc.cluster.filter.support.ConsumerContextFilter.invoke(ConsumerContextFilter.java:118)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
 at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:194)
 at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$ClusterFilterInvoker.invoke(AbstractCluster.java:92)
 at org.apache.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:103)
 at org.apache.dubbo.registry.client.migration.MigrationInvoker.invoke(MigrationInvoker.java:282)
 at org.apache.dubbo.rpc.proxy.InvocationUtil.invoke(InvocationUtil.java:56)
 at org.apache.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:75)
 at org.apache.dubbo.samples.api.GreetingServiceDubboProxy0.sayHello(GreetingServiceDubboProxy0.java)
 at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
 at com.sun.proxy.$Proxy27.sayHello(Unknown Source)
 at org.apache.dubbo.samples.action.GreetingServiceConsumer.doSayHello(GreetingServiceConsumer.java:34)
 at org.apache.dubbo.samples.ConsumerBootstrap.main(ConsumerBootstrap.java:38)

看起来像是通过路由配置找到注册的provider示例，但是我这边确实能查到启动的两个provider实例：

# kubectl describe svc dubbo-samples-xds-provider -n dubbo-demo
Name:              dubbo-samples-xds-provider
Namespace:         dubbo-xds-demo
Labels:            <none>
Annotations:       <none>
Selector:          app=dubbo-samples-xds-provider
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                None
IPs:               None
Port:              grpc  50051/TCP
TargetPort:        50051/TCP
Endpoints:         10.244.0.253:50051,10.244.0.254:50051
Session Affinity:  None
Events:            <none>

Silocean avatar Mar 15 '23 07:03 Silocean

proxyless功能没问题的

aamingaa avatar Mar 28 '23 03:03 aamingaa

我弄了一个能跑通的proxyless流量路由的脚本文件，你对照一下？https://github.com/apache/dubbo/actions/runs/4498349215/jobs/7914994051?pr=11859 。另外dubbo的grpc version要改成1.41.0，然后打个包，在dubbo-samples里引入

aamingaa avatar Mar 28 '23 03:03 aamingaa

@Silocean 请问一下后来调通了吗？是怎么解决的？

goto456 avatar Apr 25 '23 08:04 goto456

@Silocean 请问一下后来调通了吗？是怎么解决的？

还没有哈

Silocean avatar Apr 27 '23 12:04 Silocean

我弄了一个能跑通的proxyless流量路由的脚本文件，你对照一下？https://github.com/apache/dubbo/actions/runs/4498349215/jobs/7914994051?pr=11859 。另外dubbo的grpc version要改成1.41.0，然后打个包，在dubbo-samples里引入

服务网格: Proxy模式与Proxyless模式的Dubbo应用，可以互联互通？ @AlbumenJ

背景

想打通测试环境和本地开发环境的流量路由标签的环境隔离，充分利用稳定的测试环境，避免多套环境和维护成本。

Proxy模式与Proxyless模式的Dubbo应用，可以互联互通。 如官方架构图，Sidecar 与 Proxyless 模式共存

http://static.dubbo.apache.org:8080/zh-cn/overview/core-features/service-mesh/

image

刘军分享的《基于DubboMesh的标准化云原生微服务体系》和邹毅贤分享的《跨平台的统一控制面实现 Dubbo Mesh》

数据面支持多种部署形态

image image

bert82503 avatar Apr 22 '24 08:04 bert82503