AWS CodeCommit support

[carlosfunk]

Are there any plans to support AWS CodeCommit?

[rarkins]

I haven't planned on it before now, but it looks they have an API with endpoints such as CreatePullRequest, so it looks promising.

[jspenguin2017]

I have quite a few repos on CodeCommit and I would like this feature. Looking at the CodeCommit and CodePipeline documentation, I think we have most of the pieces in place.

Is there anything special I need to know or read before getting started? I checked the docs directory but I only found some notes about branch update and platform authentication.

[rarkins]

From a git point of view, we hopefully have no incompatibilities. Then it's a matter of whether the API is rich enough to support PR creation, updating, commenting, etc.

The existing platforms we have are now all in TS which should make it reasonably easy to infer requirements for a new platform from. I would start with initPlatform, getRepos, and then initRepo.

[chris2k2]

Are there any news on this?

[rarkins]

No, I’m not aware of anyone starting it. Gitea platform support was added recently so they would be a good reference for anyone wanting to add CodeCommit.

[jspenguin2017]

It turns out that I'm way more busy than I expected, so unfortunately I don't think I can start on this any time soon.

[jessieweiyi]

Any update on this?

[rarkins]

No, it likely needs some outside contributions to get it moving.

[olegkrivtsov]

I'd like to take this one.

[rarkins]

@olegkrivtsov this could be a lot of work. but at least we can start with the research side of things (working out if all the APIs we need are supported, such as issues, PRs, etc). Do you have an AWS account you can test with?

[olegkrivtsov]

Hi @rarkins ok, I can start by reviewing the AWS APIs and check what capabilities they have. Creating AWS account shouldn't be difficult.

[olegkrivtsov]

Could you please assign this to me so it's easier for me to track?

[olegkrivtsov]

Hi @rarkins I found this page describing AWS CodeCommit APIs: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CodeCommit.html

It seems that CodeCommit APIs support managing the following:

• repos
• branches
• commits
• pull requests
• merges

However, it doesn't have an integrated issue tracker. And it also seems it has some support of "labels" they call "tags", so it may be possible to label a PR.

[rarkins]

We can get by without issues if they don't have them. Please map each of these to API documentation: https://github.com/renovatebot/renovate/blob/db5137b39a472dc8e386bcbc6397e6e01d0422fa/lib/platform/types.ts#L150-L190

[olegkrivtsov]

I tried to map those functions to the API endpoints (https://docs.aws.amazon.com/codecommit/latest/APIReference):

findIssue(title: string): Promise<Issue | null>;
getIssueList(): Promise<Issue[]>;
getIssue?(number: number, useCache?: boolean): Promise<Issue>;

The above are not available. CodeCommit doesn't support issues.

getVulnerabilityAlerts(): Promise<VulnerabilityAlert[]>;

It seems that vulnerability alerts are not supported by CodeCommit.

getRawFile(fileName: string, repo?: string): Promise<string | null>;
getJsonFile(fileName: string, repo?: string): Promise<any | null>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetFile.html

initRepo(config: RepoParams): Promise<RepoResult>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetRepository.html

getPrList(): Promise<Pr[]>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_ListPullRequests.html

  ensureIssueClosing(title: string): Promise<void>;
  ensureIssue(
    issueConfig: EnsureIssueConfig
  ): Promise<EnsureIssueResult | null>;

The above are not supported by CodeCommit.

  massageMarkdown(prBody: string): string;

This is non-related to API, right?

  updatePr(prConfig: UpdatePrConfig): Promise<void>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_UpdatePullRequestDescription.html https://docs.aws.amazon.com/codecommit/latest/APIReference/API_UpdatePullRequestTitle.html

  mergePr(config: MergePRConfig): Promise<boolean>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_MergePullRequestByThreeWay.html https://docs.aws.amazon.com/codecommit/latest/APIReference/API_MergePullRequestBySquash.html https://docs.aws.amazon.com/codecommit/latest/APIReference/API_MergePullRequestByFastForward.html

  addReviewers(number: number, reviewers: string[]): Promise<void>;
  addAssignees(number: number, assignees: string[]): Promise<void>;

I don't think the API has endpoints for these. It seems that CodeCommit uses AWS IAM roles as "usernames". When we create a PR, it is assigned an ARN (IAM role ID). I'm not aware of its ability to add assignees/reviewers.

  createPr(prConfig: CreatePRConfig): Promise<Pr>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_CreatePullRequest.html

  getRepos(): Promise<string[]>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_ListRepositories.html

  getRepoForceRebase(): Promise<boolean>;

It seems this is not related to API.

  deleteLabel(number: number, label: string): Promise<void>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_UntagResource.html

  setBranchStatus(branchStatusConfig: BranchStatusConfig): Promise<void>;

As far as I could determine: https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetBranch.htm https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetCommit.html

  getBranchStatusCheck(
    branchName: string,
    context: string
  ): Promise<BranchStatus | null>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetBranch.html https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetCommit.html

  ensureCommentRemoval(
    ensureCommentRemoval:
      | EnsureCommentRemovalConfigByTopic
      | EnsureCommentRemovalConfigByContent
  ): Promise<void>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_DeleteCommentContent.html

  ensureComment(ensureComment: EnsureCommentConfig): Promise<boolean>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_UpdateComment.html

  getPr(number: number): Promise<Pr>;
  findPr(findPRConfig: FindPRConfig): Promise<Pr>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetPullRequest.html

`` refreshPr?(number: number): Promise;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetPullRequest.html

``
  getBranchStatus(branchName: string): Promise<BranchStatus>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetCommit.html

  getBranchPr(branchName: string): Promise<Pr | null>;

https://docs.aws.amazon.com/codecommit/latest/APIReference/API_GetPullRequest.html

  initPlatform(config: PlatformParams): Promise<PlatformResult>;

I guess this is not related to API.

  filterUnavailableUsers?(users: string[]): Promise<string[]>;

I think this is not supported by AWS CodeCommit.

[PhilipAbed]

I've created a POC Code that will clone + create branch + commit + create PR, and it works like a charm.

Requirements input from user:

1. repository name
2. AWS Region ( will default to us-east-1 if not set)
3. Access Key Id
4. Secret Access Key 3 and 4 are provided by AWS on the IAM user level PSB picture,

Note: according to the documentation it tells you to create an HTTP user and password for http calls which actually works for git.clone, but we don't wanna ask too much of the user so i looked for a different way to connect to http using the Access Key id and Secret Access Key, using Signature v4, which is actually more secure see: https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html for more info

Code Sample for git clone

    const localDir = 'path to your local dir where you want to clone to';
    const credentials = {
        accessKeyId: 'ACCESS_KEY_ID',
        secretAccessKey: 'ACCESS_SECRET_KEY',
        sessionToken: null
    };
    const git: SimpleGit = simpleGit(localDir, simpleGitConfig());
    const signer = new aws4.RequestSigner(
        {
            service: "codecommit",
            host: `git-codecommit.${REGION}.amazonaws.com`,
            method: "GIT",
            path: `v1/repos/${REPO_NAME}`
        },
        credentials
    );
    const username = credentials.accessKeyId;
    const password = signer.getDateTime() + "Z" + signer.signature();
    const url = `https://${username}:${password}@git-codecommit.${REGION}.amazonaws.com/v1/repos/${REPONAME}`;
    const opts: string[] = ['--filter=blob:none'];
    try {
        await git.clone(url, opts);
    } catch (e) {
        console.log(e);
    }

Code Sample for API call

    const client = new CodeCommitClient({region: ${REGION}, credentials: credentials});
    const getBranchInput:GetBranchInput = {
        branchName:'main',
        repositoryName: ${repoName}
    }
    let getBranchMain = new GetBranchCommand(getBranchInput);
    try {
        await client.send(getBranchMain);
    } catch (error) {
       // handle error
    }

API Reference: https://docs.aws.amazon.com/codecommit/latest/APIReference/Welcome.html

im planning to start implementing the platform interfaces

WDYT guys?

[rarkins]

Awesome! Can we incorporate the region as part of "endpoint"? And key/secret as username/password? Ie to avoid adding new config fields.

Best if you can clarify what new configuration you think is necessary so we can confirm.

[PhilipAbed]

Well i dont think we need any new configurations if we are going to use endpoint for region username for access key id password for secret access key

What do you mean "part of endpoint"? what do you want the endpoint to be? the full host? like endpoint = 'git-codecommit.${REGION}.amazonaws.com'?

[rarkins]

Our endpoints all start with https:// so far

[lorengordon]

Can we eliminate the requirement for an access/secret key? It should fall through to the SDK credential chain, making it compatible with aws-cli config, environment variables, or an execution role (ec2 instance profile, for example).

Also, when I clone from codecommit, I write this to my .gitconfig and let the aws-cli helper authenticate me:

[credential "https://git-codecommit.*.amazonaws.com"]
        helper = !aws codecommit credential-helper $@
        UseHttpPath = true

(fwiw, our team has been heavily involved in the Dependabot support for CodeCommit, and can confirm that this setup works there... happy to consult here also!)

[viceice]

i think that can be improved in a later PR. for first implementation it would be ok to force the manual config

[rarkins]

Should be possible to try without any username/password during initPlatform()

[PhilipAbed]

Why not both options ? I can make it optional from config and if it doesnt exist take it from enviornment.. if not then throw error

On Tue, Jul 26, 2022, 21:21 Rhys Arkins @.***> wrote:

Should be possible to try without any username/password during initPlatform()

— Reply to this email directly, view it on GitHub https://github.com/renovatebot/renovate/issues/2868#issuecomment-1195828634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBKLATSBXNUDRIRVOXCAALVWAUDBANCNFSM4GGIYEIQ . You are receiving this because you were assigned.Message ID: @.***>

[PhilipAbed]

I disagree on elimenating the config option. we dont always have the credentials in an environment .. in a docker for example renovate wouldnt have .gitconfig configured at all so --profile and cli helper arent an option.

[lorengordon]

I understand how you may have misread the suggestion, and I apologize, but the intended emphasis was on eliminating the requirement to specify the access/secret keys, not the config option.

(Also, we use .gitconfig and config envs in our docker containers that are running dependabot and it works fine, so they certainly could be an option for renovate also.)

[viceice]

(Also, we use .gitconfig and config envs in our docker containers that are running dependabot and it works fine, so they certainly could be an option for renovate also.)

.gitconfig is only an option for self-hosted renovate, as we won't ship the aws cli and don't allow to change the gitconfig from repo config

[PhilipAbed]

i'm trying to initialize repo,

1. i cant seem to find a way to get gitAuthor for aws, can we require that in config?
2. i also seem to have a problem because Initiating the Signature requires repository name and i dont have it in initRepo is it ok if i add repoName to the PlatformParams? i actually need a different git clone call for each repository that the user sets in repositories, because every repository has a different signature ill see what i can do

[viceice]

1. yes, you can validate it in initPlatform like token https://github.com/renovatebot/renovate/blob/5bbb4c60546c4b3726e2d4ed6fc7092806bf36c6/lib/modules/platform/gitea/index.ts#L181-L183

[viceice]

2. repository is passed to initRepo, so you need to compute the signature there

https://github.com/renovatebot/renovate/blob/5bbb4c60546c4b3726e2d4ed6fc7092806bf36c6/lib/modules/platform/gitea/index.ts#L235-L239

[lorengordon]

(Also, we use .gitconfig and config envs in our docker containers that are running dependabot and it works fine, so they certainly could be an option for renovate also.)

.gitconfig is only an option for self-hosted renovate, as we won't ship the aws cli and don't allow to change the gitconfig from repo config

That's not entirely correct... A user can build their own container, using the renovate container as the base. Then install aws-cli and setup the gitconfig. As long as the source code works with basic https clone urls and doesn't impose unnecessary config requirements for authentication, it works fine and is quite simple.