AWS Security Groups and Roles

[ShafiKhan18]

We are trying to evaluate Herd for data management needs on AWS. I am trying to do a demo install on AWS and the cloudformation template fails at the IAM CreateRole User step as my ID does not have the privilege to do so. Is there a way we can work around this step by having the template use an existing AWS Security group and role I can have our cloud infrastructure team create before going through the cloud formation. Being a financial services organization, there are restrictions in place due to which I am asking if this is a possibility. Please let me know.

Thanks, Shafi.

[nateiam]

Hello @ShafiKhan18 - you have a good question.

Currently the CloudFormation template performs all the steps including IAM Roles and Security Groups. We understand this creates a challenge in many organizations where there is enterprise control over specific AWS resources.

We've been talking about this a lot recently on the Herd team and are partway through an initiative to make the CloudFormation template much more flexible. We are refactoring so there are options to reference existing Roles and Security Groups -- or to have Herd create them. And we will deliver at the same time some documentation indicating required characteristics (eg what ports need to be open for the SGs). We are planning to release this in early June.

The immediate workaround is to go into the CloudFormation template and modify it to use Roles and Security Groups that your enterprise has created.

Let me know if you have more questions on this topic. If you have specific questions about the internals of the CloudFormation template, we can have an engineer answer them.

Also we are always happy to have more open-ended discussion about what you are trying to accomplish so we can help with guidance or answering questions.

[ShafiKhan18]

Hi,

            Thank you for taking the time to respond to my question.

            I like the workaround and would like to work with your engineer as I am new to Herd and Cloud. I don’t want to screw up our DEV work.

            Please feel free to reach out to me on 508-202-8884 anytime for a conversation on this.

Thanks, Shafi.

Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point you either have to stop and clear the windshield or risk everything. -Ken Orr, The Cutter Consortium

From: Nate Weisz [mailto:[email protected]] Sent: Wednesday, April 18, 2018 2:21 PM To: FINRAOS/herd Cc: Syed, Shafiullah Khan; Mention Subject: Re: [FINRAOS/herd] AWS Security Groups and Roles (#377)

Hello @ShafiKhan18https://github.com/ShafiKhan18 - you have a good question.

Currently the CloudFormation template performs all the steps including IAM Roles and Security Groups. We understand this creates a challenge in many organizations where there is enterprise control over specific AWS resources.

We've been talking about this a lot recently on the Herd team and are partway through an initiative to make the CloudFormation template much more flexible. We are refactoring so there are options to reference existing Roles and Security Groups -- or to have Herd create them. And we will deliver at the same time some documentation indicating required characteristics (eg what ports need to be open for the SGs). We are planning to release this in early June.

The immediate workaround is to go into the CloudFormation template and modify it to use Roles and Security Groups that your enterprise has created.

Let me know if you have more questions on this topic. If you have specific questions about the internals of the CloudFormation template, we can have an engineer answer them.

Also we are always happy to have more open-ended discussion about what you are trying to accomplish so we can help with guidance or answering questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FINRAOS/herd/issues/377#issuecomment-382482083, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AksRkD-AbD0DtAtO5wJytX7tXOaSgD6lks5tp4QdgaJpZM4TY1WY.

[ShafiKhan18]

Will you be able to provide any assistance?

Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point you either have to stop and clear the windshield or risk everything. -Ken Orr, The Cutter Consortium

From: Syed, Shafiullah Khan Sent: Wednesday, April 18, 2018 2:47 PM To: FINRAOS/herd; FINRAOS/herd Cc: Mention Subject: RE: [FINRAOS/herd] AWS Security Groups and Roles (#377)

Hi,

            Thank you for taking the time to respond to my question.

            I like the workaround and would like to work with your engineer as I am new to Herd and Cloud. I don’t want to screw up our DEV work.

            Please feel free to reach out to me on 508-202-8884 anytime for a conversation on this.

Thanks, Shafi.

Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point you either have to stop and clear the windshield or risk everything. -Ken Orr, The Cutter Consortium

From: Nate Weisz [mailto:[email protected]] Sent: Wednesday, April 18, 2018 2:21 PM To: FINRAOS/herd Cc: Syed, Shafiullah Khan; Mention Subject: Re: [FINRAOS/herd] AWS Security Groups and Roles (#377)

Hello @ShafiKhan18https://github.com/ShafiKhan18 - you have a good question.

Currently the CloudFormation template performs all the steps including IAM Roles and Security Groups. We understand this creates a challenge in many organizations where there is enterprise control over specific AWS resources.

We've been talking about this a lot recently on the Herd team and are partway through an initiative to make the CloudFormation template much more flexible. We are refactoring so there are options to reference existing Roles and Security Groups -- or to have Herd create them. And we will deliver at the same time some documentation indicating required characteristics (eg what ports need to be open for the SGs). We are planning to release this in early June.

The immediate workaround is to go into the CloudFormation template and modify it to use Roles and Security Groups that your enterprise has created.

Let me know if you have more questions on this topic. If you have specific questions about the internals of the CloudFormation template, we can have an engineer answer them.

Also we are always happy to have more open-ended discussion about what you are trying to accomplish so we can help with guidance or answering questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FINRAOS/herd/issues/377#issuecomment-382482083, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AksRkD-AbD0DtAtO5wJytX7tXOaSgD6lks5tp4QdgaJpZM4TY1WY.