Faster composer_lock_diff without DDEV

[beto-aveiga]

Not using DDEV to generate composer-lock-diff information was much faster in our tests.

Time comparison (with and without DDEV)

The difference was from 4 minutes to 30 seconds.

The code

This is the code we are using for that workflow:

name: "Composer Lock Diff"

on:
  pull_request:
    branches:
      # List target branches, not source branches.
      - '*'

concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

jobs:
  Composer-Lock-Diff:
    # Lock Ubuntu to always get PHP 8.1
    # See: https://github.com/shivammathur/setup-php#github-hosted-runners
    runs-on: ubuntu-22.04
    permissions:
      contents: read
      pull-requests: write

    steps:
      - uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.base.ref }}

      - uses: actions/checkout@v3
        with:
          fetch-depth: 2

      - name: Set environment variables
        run: |
          echo "PR_NUMBER=$(echo $GITHUB_REF | awk 'BEGIN { FS = "/" } ; { print $3 }')" >> $GITHUB_ENV

      - name: Install Composer
        run: |
          php --run "copy('https://getcomposer.org/installer', 'composer-setup.php');"
          php composer-setup.php --filename=composer --version=2.5.8
          php --run "unlink('composer-setup.php');"
          chmod +x composer
          mv composer /usr/local/bin/composer

      - name: Install dependencies
        run: composer install

      - name: Install composer lock diff
        run: composer global require davidrjonas/composer-lock-diff:^1.0

      - name: Get the ID of the last review (if any)
        run: |

          response=$(curl --location \
            --header "Accept: application/vnd.github+json" \
            --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}"\
            --header "X-GitHub-Api-Version: 2022-11-28" \
            https://api.github.com/repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews)

          # LAST_REVIEW_ID won't be empty when we already have a review.
          # If so, we will update the review later, instead of creating one.
          github_action_reviews=$(echo "$response" | jq '.[] | select(.user.login == "github-actions[bot]")')
          echo "LAST_REVIEW_ID=$(echo "$github_action_reviews" | jq --raw-output '.id' | tail --lines 1)" >> $GITHUB_ENV

      - name: Run composer lock diff
        run: |

          vendor/bin/composer-lock-diff \
            --from=origin/${{ github.event.pull_request.base.ref }}:composer.lock \
            --to=HEAD:composer.lock --md > composer-lock-diff.txt

          if [ $(wc --bytes < composer-lock-diff.txt) -lt 1 ]; then
              echo -e "No package differences found." > composer-lock-diff.txt
          fi

          # Generating the title with new line.
          echo '## Composer Lock Diff' > composer-lock-diff-title.txt
          echo " " >> composer-lock-diff-title.txt

          # Appending the title.
          json_body_value=$(cat composer-lock-diff-title.txt)$(cat composer-lock-diff.txt)

          # Create a JSON object with the contents in the "body" key.
          composer_lock_diff_json=$(jq --null-input --arg body "$json_body_value" '{"body": $body}')

          # New reviews must contain event:"COMMENT", otherwise the review state
          # will be "pending".
          if [ -z "$LAST_REVIEW_ID" ]; then
            composer_lock_diff_json=$(jq --null-input --arg body "$json_body_value" --arg event "COMMENT" '{"body": $body, "event": $event}')
          else
            composer_lock_diff_json=$(jq --null-input --arg body "$json_body_value" '{"body": $body}')
          fi

          # Store the JSON object back into the file.
          echo "$composer_lock_diff_json" > composer-lock-diff.json

      - name: Edit last review (if any)
        # Only for Renovate PRs.
        if: env.LAST_REVIEW_ID != '' && startsWith(github.ref, 'refs/heads/renovate/')
        run: |

          curl --location \
            --request PUT \
            --header "Accept: application/vnd.github+json" \
            --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}"\
            --header "X-GitHub-Api-Version: 2022-11-28" \
            https://api.github.com/repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews/$LAST_REVIEW_ID \
            --data @composer-lock-diff.json

      - name: Create a Review
        # Only for Renovate PRs.
        if: env.LAST_REVIEW_ID == '' && startsWith(github.ref, 'refs/heads/renovate/')
        run: |

          # Posting a review.
          curl --location \
            --request POST \
            --header "Accept: application/vnd.github+json" \
            --header "X-GitHub-Api-Version: 2022-11-28" \
            --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}"\
            https://api.github.com/repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
            --data @composer-lock-diff.json

      - name: Modify the PR description
        # Not for Renovate PRs.
        if: "!startsWith(github.ref, 'refs/heads/renovate/')"
        run: |

          source_file="pull_request.txt"
          cat composer-lock-diff.json | jq --raw-output '.body' > "composer-lock-diff.txt"
          replacement_file="composer-lock-diff.txt"
          target_file="processed.json"

          curl --fail \
            --header "Accept: application/vnd.github+json" \
            --header "X-GitHub-Api-Version: 2022-11-28" \
            --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
            https://api.github.com/repos/${{ github.repository }}/pulls/$PR_NUMBER > "$source_file"

          # Selecting body only from the JSON request with the PR info.
          cat $source_file | jq -r '.body' > "source_file.tmp"
          cp "source_file.tmp" $source_file

          # Define the start and end markers
          start_marker='<!--composer_lock_diff_begin-->'
          end_marker='<!--composer_lock_diff_end-->'

          # Check if the start marker exists in the target file
          file_content=$(<"$source_file")
          if [[ $file_content == *"$start_marker"* ]]; then
            # Replace the content between markers with the content of the replacement file
            # Read the content before the start marker
            before_start=$(sed --silent "/$start_marker/q;p" "$source_file")

            # Read the content after the end marker
            after_end=$(sed --silent "/$end_marker/,\$p" "$source_file")

            # Combine the content from file2 with the extracted parts
            new_content="$before_start\n$start_marker"
            new_content+="\n$(cat $replacement_file)"
            new_content+="\n$after_end"
            echo -e "$new_content" > $target_file
          else
              # Append the content of the replacement file to the target file
              cp "$source_file" "$target_file"
              echo "$start_marker" >> "$target_file"
              cat "$replacement_file" >> "$target_file"
              echo "$end_marker" >> "$target_file"
          fi

          # Store new PR description, escaped for JSON.
          new_description=$(cat $target_file | jq --raw-input --slurp --ascii-output)
          (echo "{\"body\": $new_description}") > "$target_file"

          # Update PR description.
          curl --fail \
            --request PATCH \
            --header "Accept: application/vnd.github+json" \
            --header "X-GitHub-Api-Version: 2022-11-28" \
            --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
            https://api.github.com/repos/${{ github.repository }}/pulls/$PR_NUMBER \
            --data @processed.json

[deviantintegral]

Nice!

I've noticed there's unfortunate race conditions with this on a project. More than once, I've been editing a PR description and this job has reverted my changes. I wonder if there's a version ID we can pass in that will cause GitHub to error out if the PR description has been modified.

I think this is also conflicting with Renovate, as I've seen updates from it where the lock diff table shows up, and others when it doesn't.

[davereid]

Related: https://github.com/Lullabot/drainpipe/issues/317

[deviantintegral]

Related: #136 .

[deviantintegral]

And #332 .

[mrdavidburns]

@beto-aveiga please review @justafish's new proposed solution https://github.com/Lullabot/drainpipe/pull/636