rocketmq icon indicating copy to clipboard operation
rocketmq copied to clipboard

add GrayRelease AllocateMessageQueue strategy

Open makabakaboom opened this issue 4 years ago • 12 comments

目前RocketMQ消费缺乏灰度发布能力,所以提交一个可以灰度发布的PR实现消费灰度标签能力

makabakaboom avatar Aug 14 '21 08:08 makabakaboom

该策略主要是将灰度Label标记在灰度实例的ClientID中,并在分配队列时,将每个Broker的指定比例的前N个队列用于所有灰度消费者来进行分配,并在发送端利用相同策略,实现灰度发送生产。

makabakaboom avatar Aug 14 '21 08:08 makabakaboom

What situation will use this gray release allocate strategy? can you give a case? people who use this ability must use this strategy before gray, and change back (or modify "percentage" both in consumer and producer)after gray finish. That is inconvenient.

ni-ze avatar Aug 17 '21 08:08 ni-ze

image

应用场景主要是用于灰度发布场景,默认没有灰度实例情况下,将所有实例进行分配。在需要业务小规模上线试用的情况下,只需要灰度部分消费实例并打上灰度标签,即可将固定比例的队列分配给灰度的实例进行消费,并在发送端通过相同算法将灰度流量发送到指定队列,待灰度验证完成后,后续可以将所有消费更新消费至新逻辑,并将灰度节点标签去掉,完成上线

makabakaboom avatar Aug 17 '21 09:08 makabakaboom

suppose there are 10 producer instances and 10 consumer instances, if we need to do gray release, how about do-gray-deployment to producer instance 1 and consumer instance 1

francisoliverlee avatar Aug 18 '21 02:08 francisoliverlee

percentage只是将队列分成灰度和正常队列的比例,如图上每个broker的16个队列将会把每个broker的前两个队列在发送和消费端均默认为灰度队列。如果我们只需要部署1个灰度消费实例时,我们只需要在该实例配置instanceName加上%GRAY%标签,或者直接通过环境变量注入方式打入灰度标签,利用CID在所有消费者具备最终一致性能力,默认所有消费者实例都能感知到有灰度实例上线,这个时候就会将灰度队列分配给带有灰度标签的消费者。而发送端利用相同算法,将带有灰度的流量直接发送到灰度的队列上,因为发送端和消费端在标识灰度队列上算法是一致的,都是每个broker的前N个队列

makabakaboom avatar Aug 18 '21 03:08 makabakaboom

Thanks for Good job ! But in the other hand, it seems like this is not a MQ's responsibility to handle our deployment requirements, keep simple and clean.

liuruinl avatar Aug 18 '21 09:08 liuruinl

Thanks for Good job ! But in the other hand, it seems like this is not a MQ's responsibility to handle our deployment requirements, keep simple and clean.

OK,不过在实际应用过程中,全链路灰度发布需求还是比较大的,而消息队列作为其中一环也应该具备该能力

makabakaboom avatar Aug 18 '21 09:08 makabakaboom

MQ itself does not have the concept or statement of grayscale. You can use different subscription to meet your needs. RocketMQ subscriptions need evolve more easily.

chenzlalvin avatar Aug 23 '21 06:08 chenzlalvin

原生灰度能力确实很有吸引力,实现方式是特殊的负载均衡策略还是不同的订阅方式还需要考虑下

lizhimins avatar Aug 31 '21 03:08 lizhimins

个人很赞成 @francisoliverlee makabakaboom 的贡献. 如果说为了保持简单和功能纯粹,那事务消息也是没有必要的.顺序消息也是没有必要的 .只需要创建多多的topic和group就能实现了,尤其事务消息,更加增加了架构扩展的复杂度(非抬杠). 我个人认为,既然好多人都在寻找如何在rocketmq中实现流量路由的功能,都在苦苦探索,有人有了方案也未必会分享出来 ,那面rocketmq为什么不能提供一套支持流量路由的功能呢? 哪怕作为插件提供出来不是也很好? (个人猜测,可能是出于商业原因,这部分特性是需要商用的)

feixueck avatar Mar 23 '22 02:03 feixueck

Multiple experimental environments need to consider.

ifplusor avatar Mar 23 '22 02:03 ifplusor

mark.

ps: 找资料刚好看到了一篇类似的文章,参考 vivo鲁班RocketMQ平台的消息灰度方案

vergilyn avatar Sep 19 '22 09:09 vergilyn

This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.

github-actions[bot] avatar Sep 20 '23 00:09 github-actions[bot]

This issue was closed because it has been inactive for 3 days since being marked as stale.

github-actions[bot] avatar Sep 24 '23 00:09 github-actions[bot]