Feature Request: Integrating a developer environment into a Mesh

[lavignes]

Developers running/testing their apps locally (i.e. using tools like docker-compose) in their own environments may want to be able to integrate with a Mesh running in a VPC.

I'm proposing we make an example of using Client VPN to add a multi-container local docker environment as a virtual node to an existing mesh.

[jfefes]

Hey there, I started a thread similar to this on reddit, thanks for making this issue!

My current use case involves having different services in App Mesh, and being able to communicate to them in a development environment without additional code changes. With the current examples around, it's fairly easy to understand how to make a few different microservices (lets call them orders customers and admin), deploy them to ECS and have them coexist within App Mesh.

However, let's say a team needs to update the orders app, which has some dependencies on customers. Instead of needing to locally run versions of both of these services, a Client VPN would be a great way to have our local orders service talk to the customers service within App Mesh without any additional code changes.

[victorlevasseur]

Hi, Are there any news on this specific issues. We are considering App Mesh for our microservice architecture but without the possibility to use it locally we don't know how we can develop our services.

[bcelenza]

Hey @victorlevasseur, we're still in the process of figuring out where we'd like to focus our efforts first.

Can you share some details on your development environment?

1. What tools do you use for development?
2. Would you want to run the whole mesh or just part of it in a dev environment?
3. If just part of a mesh, can you share some details on how you'd want connectivity to flow between the local and remote components?

I'm also going to turn this into a proper feature request for the App Mesh roadmap and move it into that repository.

[nathanpeck]

I like this feature request. I'd say the gold standard for experience here would be something like telepresence: https://www.telepresence.io/

Provide a toolchain that makes it easy to launch a proxy in the mesh inside my AWS VPC, exposed to the public internet so it can be connected to a local proxy on my desktop. Then I can run my container on my local machine, it talks to a local Envoy which talks to the remote inside my VPC, which can send traffic to a real production virtual node

[janaka]

In our current set up, when an engineer runs a microservice on their laptop in debug mode, it gets registered as a node in service discovery in our test dev environment in AWS. So from service discovery and interservice comms point of view the instance of the app on the devs machine is treated the same as all other instances of microservices running in the dev test environment in the AWS VPC. The engineer would be on the corp VPN (or physically on the office network) for the network path to work. Today we use Eureka for service discovery and RPC over HTTP for interservice comms.

Ideally if/when we move to App Mesh and CloudMap we'd want to be able to provide a similar experience. To do that we probably need to be able to register a locally running app (native or in a container) with Cloud Map and App Mesh, be able to have App Mesh manage the local Envoy instance, and of course network traffic routing. I'm probably missing some details here but I think those are the basics. I don't know if this is currently possible at all because we haven't got as far as trying yet. But ideal AWS would provide something out of the box. Providing a decent local development experience is something AWS hasn't done a good job of generally IMO so hopefully the App Mesh team will take some steps to improve that situation.

Our engineering team are on a mix of macOS and Windows with a sprinkle of Linux.

[apottere]

This is probably the biggest blocker for us adopting AppMesh for our microservices - to the point that we might just move all of our ECS services into EKS to take advantage of telepresence. Almost all of our services are dependent on other services and data sources that we don't want to or can't run locally. Right now we use a VPN and do "service discovery" by putting ALB hostnames in Parameter Store, which means our local applications can resolve and call applications running in our dev environment.

If I understand correctly, it should be possible to write a script that sets up a local Envoy instance, connects it (through some sort of VPN) to AppMesh or an Envoy instance in AWS connected to the mesh, and use that to somehow proxy connections for our local applications. Figuring out how to set that up is another thing entirely. It seems like the aws cli should have a telepresence-esque command that can do that all for you.

[ssypi]

@apottere Did you find a good solution for this? I've been testing running a socks proxy through SSH connection to an EC2 instance that is running envoy connected to the mesh. It feels a bit dirty solution, but it kinda works. We then use proxychains to route all traffic from the locally running dev process through the proxy (eg. "proxychains npm start") and can do dns resolution and connecting to mesh services through it. If needed you can also run ssh remote port forwarding to enable other mesh services to connect to the locally running dev server. Just wondering what you ended up with and whether you came up with anything better.

Ofcourse the developer could always just mock or run the services they need to connect to locally, but it can get quite complex quickly and in some cases it is just easier to run a complete development or staging mesh in AppMesh and connect to it from the local environment.

[apottere]

@ssypi No, we never adopted AppMesh because we didn't feel it was worth the effort to figure this out if EKS was on our roadmap anyway.

[rndrr]

+1

My organization's (500+ engineers) local dev workflow is similar to @janaka's (local app(s) integrated with a remote test env on AWS) so this is a blocker for us. As much as we'd like to go all-in with App Mesh, we'll be forced to choose a service mesh solution with a better local development story (e.g. Consul or Envoy).

[EladG77]

Any news about this?

[paulohenriqu]

Any news about that? A telepresence like solution would be great. That's blocking us also to adopt AppMesh.