azure-service-bus
azure-service-bus copied to clipboard
Azure
Local development story?
Trying to get an understanding of what the local development story is. RabbitMQ for example can be installed on a dev machine, or run as a Docker container. If I have development/build images that need Azure Service Bus, what's the story of getting a runnable instance locally?
There's no story Jimmy. Either MSDN or another Azure subscription can be used to run on the real thing.
So what's the story there then? Each developer creates their own namespace? Picks an option that keeps them under the MSDN limit?
Or runs a shared namespace amongst the team? That would run into all the normal problems of shared dev resources.
Either way works.
Shared namespace is fine if you can "sandbox" development. I always advise to leverage MSDN subscription, unless you plan to use premium. Then you don't want create that namespace and keep around if not used. Depending on your scenario I'd have other options, but you know, "it depends" 🙂
Or, abstract your service bus layer, and run whatever you want locally?
∞ Andy Badera
On Wed, Aug 15, 2018 at 9:39 AM, Sean Feldman [email protected] wrote:
Either way works.
Shared namespace is fine if you can "sandbox" development. I always advise to leverage MSDN subscription, unless you plan to use premium. Then you don't want create that namespace and keep around if not used. Depending on your scenario I'd have other options, but you know, "it depends" 🙂
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-service-bus/issues/223#issuecomment-413218586, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAzzIzJgTGFjCOzulz8FJui2UdFLfuvks5uRDKzgaJpZM4V-M3W .
Layer of abstraction for code, yes. Running code w/o exercising client/broker, no. When working with ASB, you really should experience the "real thing".
How is an abstraction layer any different? Develop locally against whatever you want, run your automated integration tests against ASB. At a developer level, it shouldn't matter.
∞ Andy Badera
On Wed, Aug 15, 2018 at 10:09 AM, Sean Feldman [email protected] wrote:
Layer of abstraction for code, yes. Running code w/o exercising client/broker, no. When working with ASB, you really should experience the "real thing".
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-service-bus/issues/223#issuecomment-413227824, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAzzME33C1xxdJ_xe5-hPCdTyDLsCN8ks5uRDmRgaJpZM4V-M3W .
@andrewbadera and when integration test (if you have one) against ASB is failing, what are you going to do locally? Wouldn't you switch back to ASB from whatever you were running against? And now you have to deal with the "perks" of two different technologies.
@andrewbadera don't get me wrong. I am not against layer of abstraction. I'm not a fan of replacing ASB with some other technology to emulate it.
Yeah, there are real differences between different messaging transports that directly affect development. I wouldn't want to swap for example ASB for "messages are files in a folder"
This question is probably related to this feedback where @SeanFeldman also commented.
We do have the ability to run everyting dotnet core, redis, functions, sql server, storage and cosmosdb locally, but not servicebus, hence the number of votes on the feedback page. And the Service Bus On-Prem option suggested in the feedback doesn't really seem to be a in-sync option
Yeah, just gonna put my voice here as well. No option isn't really a great selling point. This has downstream effects too with EventHubs which are incredibly useful, but at least I can simulate them by using Kafka protocol enabled hubs, but that pushes me away from functions. My option here is to have every dev in my org use their MSDN account to setup dev resources? Very tempting reason to not use Service Bus and go to Rabbit.
It seems like utter nonsense to not be able to develop and run integration tests locally. At present I have to strip out and service bus bindings to test.
@arjunsol Can you use RabbitMQ for localhost dev and switch over to ASB for staging/prod ? Absolute PITA but ... a workaround.
@PureKrome do you mean for mocking purpose? That mean 2 different implementations to maintain, right?
@ggirard07 localhost scenario == rabbitmq is like an emulator. It's not a mock as in .. the code does literally interact with it. It's a dependency. staging/prod == ASB. so again, it's a dependency.
code has some abstraction for suscribe/publish and your application decides to leverage either RabbitMQ or ASB for your IEventBus.
That mean 2 different implementations to maintain, right?
Yes. Which is why i'm screaming internally. But when you have to ship something, you gotta work with the tools around you. So to me, switching between the two is far from ideal BUT it works. I got this from MS's NET-Microservices-Architecture-for-Containerized-NET-Applications-(Microsoft-eBook).
Also, I tried asking ClemensV about this on twitter a few days ago. Either I failed to communicate my question or we have a different opinion, on the matter.
... especially when there's emulators and images for Azure Storage / Sql or NoSql db's / Redis / Elastic Search / etc..
SIDE NOTE: also like how Udi Dahan said they use Sql Server as their EB for local/test with NServiceBus.SqlServer as their transport/too.
cc @jbogard - Can you fill us in, with how you/your team handled this scenario, plz?
It would probably be a lot closer to use a localhost AMQP1.0 broker for the mock @PureKrome ? Unless you were already thinking of the AMQP1.0 support in RabbitMQ server. There's another twitter thread discussing this topic here: https://twitter.com/jeremydmiller/status/1080496980168712192
It would probably be a lot closer to use a localhost AMQP1.0 broker
Um ... er .. all i've researched is using RabbitMQ and have no idea (yet) what this means with resepct to protocols/etc. All this time spent is being wasted (unfortunatly) getting the tooling and abstractions to work ... just to get a localhost dev story up-n-running. Man - this is 2019 and this should be a solved problem already :( These tools have been around for a while now. So much frustration :(
All I know is this:
- We wish to leverage ASB for production. Code to do use ASB is nice and easy.
- Do not wish to use ASB for localhost. No, we're not a plane. Now we need to use RabbitMQ (docker image + does same high level functionality as ASB). Now we also need to abstract over ASB and RMQ.
so - frustrating :(
@PureKrome
I had thought of that, we're using FunctionMonkey to cut down on boiler plate. I'm unsure if I can make it play nicely.
https://functionmonkey.azurefromthetrenches.com/
It's possible to trigger a service bus trigger via http using the /admin route, (https://docs.microsoft.com/en-us/azure/azure-functions/functions-manually-run-non-http) though I haven't quite got the hang of serializing to a BrokeredMessage, this would be step 1 for me, as I could then just trigger my function via postman when running locally.
The second issue is that if you have a service bus triggered function and you don't provide a connection string the trigger will blow up. If there was a way round this that would be great. Maybe being able to switch out the type of trigger via configuration or suppressing the exception, but not sure if that's possible. Anyone else find a way of doing that?
My accompanying Twitter thread.
Back in 2016/2017, I looked into a local emulation option for service bus as the company I was working at was in a position to need something like it. Today, I find myself in a similar situation again as we don't have the finances to waste, arbitrarily provisioning cloud resources for each developer and need a consistent and repeatable local development experience.
I didn't really get into it much back then, but feeling a bit more motivated now, I'm sadly seeing things that are a bit off-putting about how badly the message is being missed. I'm going to recount my experience on this, I realize some of it will be reiterating references already mentioned. But by the end, my hope is that you'll understand why some could be frustrated by how this is being handled, how inconsistent it is and how false and even dismissive some of the official responses are coming off...
The first result most tend to find is this stackoverflow question which has an answer that's been accepted.
The answer points to this feedback forum question from 2012 with at the time of this comment, 149 votes. What galls me about the response is its sheer combined temerity and inaccuracy:
It has been seven years since 2012. The interest on this uservoice is strong enough to warrant further investigation, yet here's a followup with 68 votes of its own.
Today, I decided to ask again after taking the time to do a bit of light research, collating the signals actual users and the community have been trying to send to the service bus team.
The response?
Similar to a prior poster in this thread, when @clemensv was paged into the discussion by @cmatskas, the response was unhelpful and really, seemed to talk past me. It's a little less great in the prior thread from @PureKrome which even seems to get a bit condescending...
While I disagree with this quip on the basis of my own use case, all that can be considered beside the point - it again talks right past the concerns on the basis of a singular ideal.
I'm not sure what the service bus team is waiting for on this one or what would qualify as sufficient. Certainly, a lot has been done to dilute, defer, confuse or even outright dismiss demand for this feature from multiple inquiries fielded seemingly out of hand in multiple forums.
From what I'm reading, I think when it comes to a local service bus emulator, it's definitely at a point that warrants action. If it really and truly is the worst idea possible in this context, do us all a favour and put a clear and official explanation in the service bus documentation as to why we shouldn't want a local emulator and what our alternative should be on Azure, specifically for topics.
Finally, I also think the responses so far are inconsistent both as they pertain to my perception of a modern-Microsoft, but also what many people value as a first class, competitive option.
"No local dev for anything of serious complexity"... hmm... Kafka can be hosted in a container.
Maybe can someone give us a technical explanation why this is not possible, or maybe they want to be so.
A nice little update today...
Sending good thoughts in the hopes that they find a way to make this happen soon!
This is the 3rd time/company have been looking in to this very thing. How to do local development and integration testing with ASB. Every other part of the stack can be run locally in a container, except this. I would be great to get something like Azurite that is API compatible and containerized.
It would be very useful to have this when working offline or as part of our unit testing/integration testing systems.
This just amazes me.
I don't know how many times I have been in a disconnected situation (e.g. 14 hour plane flight overseas) where I would love to get some of my development work done but can't because somebody, somewhere decides to take that kind of a stance.
Can I work around it or on something besides the ASB part? Sure. But why should I have to be told what I can and can't work on based on my connected status.
It would be really really helpful to have an Azure service bus emulator.
I came across this problem while trying to create a cloud independent test environment. This really gets in our way as a software developer, and I think using RabbitMQ is not a good alternative, considering that the idea of local testing is to simulate a real scenario. Anyway, I hope a solution arrives soon!
