Remove list of hard-coded region names

[aks]

Why does ex_aws have a hard-coded list of region names? That's silly.

AWS itself should be the Source of Truth on what region names are acceptable or not.

As soon as a name is hard-coded, it is likely (possible) to be out-of-date, or missing names that are acceptable. Either way, the SDK becomes less useful.

Remove the hard-coded list, trust that the client knows the right names, and trust that the AWS service will respond appropriately when the incorrect region name is used.

[samuelgoodell]

Considering how many issues on this repository are due to unsupported regions, this seems like a no-brainer move. If the maintainers are distant, I'm happy to take a shot at this. Will check back in a week and if no update make a PR.

[bernardd]

So I've got a few things to say on this:

the maintainers

Maintainer, singular. There's only one of me, and I only took over the project fairly recently. Help is always welcome.

Why does ex_aws have a hard-coded list of region names?

Because the way we get metadata about things like region-specific hostnames and supported protocols on various services relies on it. It may surprise you to learn that AWS is not a big, homogenous blob which supplies completely identical services from one region to another. There are variations and we have to deal with them.

That's silly.

Perhaps, but it's silliness that is also used by botocore, AWS's own Python bindings library. In fact if you look at the endpoints.json file in there, it may look familiar. Now it may well be that AWS are coding own service libraries in the dumbest possible way, but that would not be my first guess about what's going on.

I'd also note that criticising a design decision without making any attempt to understand why it might have been made is perhaps not the greatest way in the world to go about getting someone to do work for you.

Now, having said all that:

Considering how many issues on this repository are due to unsupported regions

I agree that it's far from ideal and I agree that it creates work , both for users and for me.

this seems like a no-brainer move

It would be, if there was a no-brainer solution. However it's not immediately clear to me what that is. ("Remove the hard-coded list" is not a solution in and of itself, for the reasons above).

My first idea would be to pull in the latest, official, endpoints.json file from botocore during the compilation process and transform that (or include it directly, copyright permitting) into our endpoints.exs file. That would keep ex_aws more or less up to date with AWS's official libraries, and probably solve most of the issues.

I'm more than happy to see other suggestions and/or PRs.

[ericmj]

@bernardd First off, thanks for taking over the maintenance of ex_aws 💜

My first idea would be to pull in the latest, official, endpoints.json file from botocore during the compilation process and transform that (or include it directly, copyright permitting) into our endpoints.exs file. That would keep ex_aws more or less up to date with AWS's official libraries, and probably solve most of the issues.

Sourcing the file from an official supported project sounds like a good idea, but I would recommend against fetching the file during compilation. It can be a bad developer experience if the library cannot compile because there is no network connectivity.

We have a similar problem in castore where we need to source the list of certificates from Mozilla. It is important that the list is up to date and that it is easy to update so it's as little manual work as possible for the maintainers so we do not drag our feet before doing it.

We have two internal mix tasks that check if our certificate store is out of date mix certdata --check-outdated and another that updates it mix certdata https://github.com/elixir-mint/castore/blob/master/lib/mix/tasks/certdata.ex. There is a nightly CI job https://github.com/elixir-mint/castore/blob/master/.github/workflows/outdated.yml that fails if the store is out of date and notifies us. For security reasons we do not update the store in the CI job, but that would certainly be possible as well.

[bernardd]

Thanks @ericmj, those are excellent points and suggestions. I particularly like the idea of a nightly CI job to take care of it.

Out of curiosity, could you elaborate on the security issues that mean you don't make it a full automated process? I suspect they may be specific to the kind of work you're doing with certificates and not applicable to this case, but it'd be good to know for sure.