[ECR] [Replication]: EventBridge events that fire when an image is replicated

[michaelb990]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request I'd like to get EventBridge events when an image is replicated. These events should fire in the source registry (after the final destination completes) & the destination registry (for each destination).

Which service(s) is this request for? ECR

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? I'd like to build workflows that trigger once the image is available in a destination region or finish a job in a source region once all regions have been replicated to.

Are you currently working around this issue? N/A.

Additional context This issue was filed by the ECR team. Please let us know if you have a use case for this feature / help us prioritize.

Attachments If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[drobbins-ancile]

Would this be required for CodePipeline ECR Source action updates?

I've created a pipeline (Account B) in CodePipeline with an ECR source action set to a specific tag and it is not starting pipeline execution when an image with that tag is replicated from another account (Account A).

[michaelb990]

@drobbins-ancile yes, the CodePipeline ECR Source action is triggered via an EventBridge rule. Unfortunately, because we don't currently emit events when an image is created / updated via replication, they will not trigger actions in CodePipeline.

We're working on adding this kind of an event and will keep this use case in mind. I believe it will require changes on the CodePipeline side as well, as this will probably be a different event action-type. Thanks for letting us know!

[drobbins-ancile]

@drobbins-ancile yes, the CodePipeline ECR Source action is triggered via an EventBridge rule. Unfortunately, because we don't currently emit events when an image is created / updated via replication, they will not trigger actions in CodePipeline.

We're working on adding this kind of an event and will keep this use case in mind. I believe it will require changes on the CodePipeline side as well, as this will probably be a different event action-type. Thanks for letting us know!

Any updates or an ETA for this?

[adamcousins]

@michaelb990 Can you confirm the event 'ReplicateImage' is the implementation for this issue? In addition is this event generated upon the final layer of the image or can be triggered upon each layer push within the one image udring during the replication cycle?

[adamcousins]

After further investigate on my above messages it seems the 'ReplicateImage' event doesnt contain any information about the destination image id (even though would be the same). I would like to consume an event on the destination registry and trigger codepipeline.

[michaelb990]

We are planning to emit this event after the image is fully replicated (all layers + manifest), which would be equivalent to the behavior of the event that's emitted after an image is pushed.

As far as the information included in the event, I am not sure where you're looking at the ReplicateImage event, but we have not finalized a specific schema yet. Our current thinking is to include most of the same information in the detail fields as we do in the source region's push event (image digest/tag, repository name) plus some additional replication-specific fields.

Unfortunately, we still don't have an ETA to share for this.

[marsadle]

Would it be possible to at least have the normal "push completed" event emitted?

As much as replication makes it easier on paper to deploy an image to multiple AWS accounts and regions, it doesn't actually, because automation based on image updates can't be properly used.

[adamcousins]

@michaelb990 Was this feature just released with the recent changes to replication options? I didnt see it in the blog or whats new entry

[abhijitvk]

@michaelb990 Is there any workaround on the replicated destination AWS account CodePipeline to trigger the build for image replication? It works in the primary account (DevOps) but doesn't trigger in the replicated AWS account (prod)

[drobbins-ancile]

@michaelb990 Is there any workaround on the replicated destination AWS account CodePipeline to trigger the build for image replication? It works in the primary account (DevOps) but doesn't trigger in the replicated AWS account (prod)

The only solution I've been able to come up with is a lambda as the final step in the primary pipeline that starts the replicated account pipeline.

[abhijitvk]

Thank you @drobbins-ancile for your suggestion. Do you mind sharing a code snippet of the CodePipeline config and lambda function that starts the account pipeline? So if I have multiple pipelines, can the lambda function trigger corresponding replicated account pipeline?

[drobbins-ancile]

Thank you @drobbins-ancile for your suggestion. Do you mind sharing a code snippet of the CodePipeline config and lambda function that starts the account pipeline? So if I have multiple pipelines, can the lambda function trigger corresponding replicated account pipeline?

It can. It just needs the required permissions to make the cross-account API call.

[abhijitvk]

Thank you @drobbins-ancile for your suggestion. Do you mind sharing a code snippet of the CodePipeline config and lambda function that starts the account pipeline? So if I have multiple pipelines, can the lambda function trigger corresponding replicated account pipeline?

It can. It just needs the required permissions to make the cross-account API call.

Found another way to trigger the code pipeline via the CloudWatch event rule in the replicated account. Example event pattern as below with the target set as CodePipeline ARN

{ "source": [ "aws.ecr" ], "detail": { "image-tags": [ "latest" ], "repository-name": [ "image_prefix/image_name" ], "scan-status": [ "COMPLETE" ] }, "detail-type": [ "ECR Image Scan" ] }

[tedivm]

Is there any update on the timeline for this? It's a pretty critical feature to be missing.

[rabidscorpio]

Now that EventBridge is available cross-region with every region except China and Gov, one way to do this would be to trigger a cloudwatch event on a successful ECR push that will start a workflow in the replicated region. What's not clear yet is how long replication takes once a successful push happens and if any lag would cause the triggered workflow to fail (I haven't tested this out yet).

[drobbins-ancile]

Now that EventBridge is available cross-region with every region except China and Gov, one way to do this would be to trigger a cloudwatch event on a successful ECR push that will start a workflow in the replicated region. What's not clear yet is how long replication takes once a successful push happens and if any lag would cause the triggered workflow to fail (I haven't tested this out yet).

This is essentially what we're doing now but it's certainly not ideal. So far we haven't seen the cross-region EventBridge notification trigger a pipeline before the ECR image has been replicated but it could definitely happen.

This issue has been open for 15 months.

[har26399]

Hi, are there any updates on the timeline for this issue?

[baonguyen84]

Thank you @drobbins-ancile for your suggestion. Do you mind sharing a code snippet of the CodePipeline config and lambda function that starts the account pipeline? So if I have multiple pipelines, can the lambda function trigger corresponding replicated account pipeline?

It can. It just needs the required permissions to make the cross-account API call.

Found another way to trigger the code pipeline via the CloudWatch event rule in the replicated account. Example event pattern as below with the target set as CodePipeline ARN

{ "source": [ "aws.ecr" ], "detail": { "image-tags": [ "latest" ], "repository-name": [ "image_prefix/image_name" ], "scan-status": [ "COMPLETE" ] }, "detail-type": [ "ECR Image Scan" ] }

@abhijitvk Encountering same issue here and try your suggestion but did not see any event emit in the rule monitoring ... Not sure if there is anything change on the CW. May I ask if you are still have this working, then? TY

[christian-deleon]

I'm also wondering if this is still in the works?

[drobbins-ancile]

Thank you @drobbins-ancile for your suggestion. Do you mind sharing a code snippet of the CodePipeline config and lambda function that starts the account pipeline? So if I have multiple pipelines, can the lambda function trigger corresponding replicated account pipeline?

It can. It just needs the required permissions to make the cross-account API call.

Found another way to trigger the code pipeline via the CloudWatch event rule in the replicated account. Example event pattern as below with the target set as CodePipeline ARN { "source": [ "aws.ecr" ], "detail": { "image-tags": [ "latest" ], "repository-name": [ "image_prefix/image_name" ], "scan-status": [ "COMPLETE" ] }, "detail-type": [ "ECR Image Scan" ] }

@abhijitvk Encountering same issue here and try your suggestion but did not see any event emit in the rule monitoring ... Not sure if there is anything change on the CW. May I ask if you are still have this working, then? TY

Yes this still works for us. Below is a code snippet from one of our pipelines where this is working:

  EcrSourceRule:
    Type: AWS::Events::Rule
    Properties: 
      Name: !Sub ECR-Source-Action-Rule-${ServiceName}-${EnvironmentIdentifier}
      EventPattern:
        source:
          - aws.ecr
        detail-type:
          - ECR Image Action
        detail:
          action-type:
            - PUSH
          result:
            - SUCCESS
          repository-name:
            - !Ref EcrRepo
          image-tag:
            - !Ref ImageTag
      Targets: 
        - Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:Fargate-Deployment-Pipeline-${ServiceName}-${EnvironmentIdentifier}
          RoleArn: !GetAtt CloudWatchEventsRole.Arn
          Id: Fargate-Deployment-Pipeline

  CloudWatchEventsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement: 
          - Effect: "Allow"
            Principal: 
              Service: 
                - "events.amazonaws.com"
            Action: 
              - "sts:AssumeRole"
      Description: Allow CloudWatch Events to trigger codepipeline
      Policies:
        - PolicyName: Allow-Pipeline-Execution
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - iam:PassRole
                  - codepipeline:StartPipelineExecution
                Resource: "*"

[drobbins-ancile]

Any update on this? It's been 2 years.

[sftim]

A use case for this: trigger a Step Function that then triggers creation of a (Kubernetes) Job that prefetches the image layers onto nodes matching a selector.

[FANMixco]

Any news? This worked for us:

Replaced this event in the event bridge

//Push
{
  "source": ["aws.ecr"],
  "detail": {
    "action-type": ["PUSH"],
    "image-tag": ["latest"],
    "repository-name": ["YOUR_IMAGE"],
    "result": ["SUCCESS"]
  },
  "detail-type": ["ECR Image Action"]
}

By this:

//Replicate
//image-tags instead image-tag
//remove result
//add scan-status
{
  "source": ["aws.ecr"],
  "detail": {
    "image-tags": ["latest"],
    "repository-name": ["YOUR_IMAGE"],
    "scan-status": ["COMPLETE"]
  },
  "detail-type": ["ECR Image Scan"]
}

It's pretty much the same from @abhijitvk, I just added some small comments where the trick is. Thanks @abhijitvk!