synthetics-ci-github-action
synthetics-ci-github-action copied to clipboard
DataDog
Getting "Argument list too long" when `raw-results` output is too big
If we run too many tests the raw-result output gets too large for Github Actions to handle it. It cannot be used in any following step. All things we tried failed with the error Argument list too long.
Example worflow:
name: Test
on:
pull_request:
paths:
- '.github/workflows/test.yml'
jobs:
test:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Run Datadog Synthetic tests
id: run-synthetics-tests
uses: DataDog/synthetics-ci-github-action@676b9d61f8118d2fa134d964d56151714d755c0a # v3.8.1
with:
datadog-site: datadoghq.eu
api-key: ${{secrets.DATADOG_API_KEY}}
app-key: ${{secrets.DATADOG_APP_KEY}}
test-search-query: tag:quality-gateway tag:"env:development"
- name: Example of using outputs
env:
RAW_RESULTS: ${{ steps.run-synthetics-tests.outputs.raw-results }}
run: |
echo 'Batch URL: ${{ steps.run-synthetics-tests.outputs.batch-url }}'
echo 'Raw Results:'
echo $RAW_RESULTS | jq '.'
Log:
2025-11-05T12:14:16.8745148Z Current runner version: '2.329.0'
2025-11-05T12:14:16.8776897Z ##[group]Runner Image Provisioner
2025-11-05T12:14:16.8778360Z Hosted Compute Agent
2025-11-05T12:14:16.8778968Z Version: 20251016.436
2025-11-05T12:14:16.8779671Z Commit: 8ab8ac8bfd662a3739dab9fe09456aba92132568
2025-11-05T12:14:16.8780389Z Build Date: 2025-10-15T20:44:12Z
2025-11-05T12:14:16.8781006Z ##[endgroup]
2025-11-05T12:14:16.8781637Z ##[group]Operating System
2025-11-05T12:14:16.8782196Z Ubuntu
2025-11-05T12:14:16.8782692Z 24.04.3
2025-11-05T12:14:16.8783205Z LTS
2025-11-05T12:14:16.8783679Z ##[endgroup]
2025-11-05T12:14:16.8784234Z ##[group]Runner Image
2025-11-05T12:14:16.8784844Z Image: ubuntu-24.04
2025-11-05T12:14:16.8785361Z Version: 20251030.96.2
2025-11-05T12:14:16.8786441Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20251030.96/images/ubuntu/Ubuntu2404-Readme.md
2025-11-05T12:14:16.8788373Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20251030.96
2025-11-05T12:14:16.8789439Z ##[endgroup]
2025-11-05T12:14:16.8790521Z ##[group]GITHUB_TOKEN Permissions
2025-11-05T12:14:16.8792664Z Contents: read
2025-11-05T12:14:16.8793317Z Metadata: read
2025-11-05T12:14:16.8793841Z ##[endgroup]
2025-11-05T12:14:16.8796104Z Secret source: Actions
2025-11-05T12:14:16.8797391Z Prepare workflow directory
2025-11-05T12:14:16.9212598Z Prepare all required actions
2025-11-05T12:14:16.9266582Z Getting action download info
2025-11-05T12:14:17.2410242Z Download action repository 'DataDog/synthetics-ci-github-action@676b9d61f8118d2fa134d964d56151714d755c0a' (SHA:676b9d61f8118d2fa134d964d56151714d755c0a)
2025-11-05T12:14:18.1872422Z Complete job name: test
2025-11-05T12:14:18.2562655Z ##[group]Run DataDog/synthetics-ci-github-action@676b9d61f8118d2fa134d964d56151714d755c0a
2025-11-05T12:14:18.2564127Z with:
2025-11-05T12:14:18.2564583Z datadog-site: datadoghq.eu
2025-11-05T12:14:18.2565232Z api-key: ***
2025-11-05T12:14:18.2565764Z app-key: ***
2025-11-05T12:14:18.2566351Z test-search-query: tag:quality-gateway tag:"env:development"
2025-11-05T12:14:18.2567214Z tunnel: false
2025-11-05T12:14:18.2567953Z ##[endgroup]
2025-11-05T12:14:19.6230299Z [mqj-rrp-26u] Found test "Test on wrong example.com/api/mobileapp/puc_de/login"
2025-11-05T12:14:19.6235573Z [vh3-6j3-hh2] Found test "[Quality Gate - Development] PUC_NL Integration Path"
2025-11-05T12:14:19.6252051Z [ynm-97u-azm] Found test "Gift card store APIs integration test [development] (Terraform generated)" (non-blocking)
2025-11-05T12:14:19.6289653Z [yri-8s3-6vq] Found test "[Buy] DE Checkout with Credit Card Payment [Development]"
2025-11-05T12:14:19.6368831Z [mby-vm7-emh] Found test "[Quality Gate - Development] PUC_NL Integration Path (cloned)"
2025-11-05T12:14:19.6374776Z [578-2xh-7c8] Found test "[Buy] NL Checkout with iDeal Payment [Development]"
2025-11-05T12:14:19.6475752Z [ycq-y9y-cqb] Found test "[Buy] DE Wishlist Test [Development]"
2025-11-05T12:14:19.6484059Z [v34-esy-zjx] Found test "[BUY] Complete member checkout flow with registration [Development]"
2025-11-05T12:14:19.6566346Z [yy7-df3-ncx] Found test "Test Customer Data and refresh token after a login in Development"
2025-11-05T12:14:19.6575399Z [t59-9kp-fre] Found test "Search via Landing Page works"
2025-11-05T12:14:19.6772623Z [tvp-p4z-2gc] Found test "Test register on development"
2025-11-05T12:14:19.6777488Z
2025-11-05T12:14:19.6779259Z
2025-11-05T12:14:20.0678813Z View pending summary in Datadog: https://app.datadoghq.eu/synthetics/explorer/ci?batchResultId=d22af079-0d54-415a-ba74-6d973c2d04c6
2025-11-05T12:14:20.0680322Z
2025-11-05T12:14:20.0793377Z - Waiting for 11 tests (ynm-97u-azm, t59-9kp-fre, yy7-df3-ncx, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, yri-8s3-6vq, ycq-y9y-cqb, 578-2xh-7c8, vh3-6j3-hh2, …)…
2025-11-05T12:14:20.0794198Z
2025-11-05T12:26:57.4936745Z ✓ [vh3-6j3-hh2] [Quality Gate - Development] PUC_NL Integration Path - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:26:57.4938158Z • Total duration: 75733 ms - View test run details:
2025-11-05T12:26:57.4940145Z ⎋ https://app.datadoghq.eu/synthetics/details/vh3-6j3-hh2/result/6758864104527414269?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:26:57.4941021Z
2025-11-05T12:26:57.4942043Z - Waiting for 10 tests (ynm-97u-azm, t59-9kp-fre, yy7-df3-ncx, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, yri-8s3-6vq, ycq-y9y-cqb, 578-2xh-7c8, mby-vm7-emh)…
2025-11-05T12:26:57.4942994Z
2025-11-05T12:28:17.2152727Z ✓ [t59-9kp-fre] Search via Landing Page works - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:28:17.2153756Z • Total duration: 67241 ms - View test run details:
2025-11-05T12:28:17.2155003Z ⎋ https://app.datadoghq.eu/synthetics/details/t59-9kp-fre/result/1296064796772006286?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:28:17.2156170Z
2025-11-05T12:29:37.8043662Z ✓ [t59-9kp-fre] Search via Landing Page works - location: Frankfurt (AWS) - device: chrome.laptop_large
2025-11-05T12:29:37.8067558Z • Total duration: 66786 ms - View test run details:
2025-11-05T12:29:37.8069393Z ⎋ https://app.datadoghq.eu/synthetics/details/t59-9kp-fre/result/7596875305522495851?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:29:37.8070662Z
2025-11-05T12:29:37.8071534Z - Waiting for 9 tests (ynm-97u-azm, yy7-df3-ncx, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, yri-8s3-6vq, ycq-y9y-cqb, 578-2xh-7c8, mby-vm7-emh)…
2025-11-05T12:29:37.8072339Z
2025-11-05T12:30:47.0221692Z ✓ [578-2xh-7c8] [Buy] NL Checkout with iDeal Payment [Development] - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:30:47.0238713Z • Total duration: 37676 ms - View test run details:
2025-11-05T12:30:47.0242282Z ⎋ https://app.datadoghq.eu/synthetics/details/578-2xh-7c8/result/1504027356690906268?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:30:47.0243215Z
2025-11-05T12:30:47.0254489Z - Waiting for 8 tests (ynm-97u-azm, yy7-df3-ncx, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, yri-8s3-6vq, ycq-y9y-cqb, mby-vm7-emh)…
2025-11-05T12:30:47.0266341Z
2025-11-05T12:32:07.5855105Z ✓ [ycq-y9y-cqb] [Buy] DE Wishlist Test [Development] - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:32:07.5856656Z • Total duration: 67695 ms - View test run details:
2025-11-05T12:32:07.5869064Z ⎋ https://app.datadoghq.eu/synthetics/details/ycq-y9y-cqb/result/6152827080215222718?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:32:07.5870111Z
2025-11-05T12:32:07.5870747Z - Waiting for 7 tests (ynm-97u-azm, yy7-df3-ncx, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, yri-8s3-6vq, mby-vm7-emh)…
2025-11-05T12:32:07.5871446Z
2025-11-05T12:34:10.0091836Z ✓ [yri-8s3-6vq] [Buy] DE Checkout with Credit Card Payment [Development] - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:34:10.0103350Z • Total duration: 89879 ms - View test run details:
2025-11-05T12:34:10.0104791Z ⎋ https://app.datadoghq.eu/synthetics/details/yri-8s3-6vq/result/5601172190915288332?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:34:10.0105769Z
2025-11-05T12:34:10.0106423Z ✓ [yy7-df3-ncx] Test Customer Data and refresh token after a login in Development - location: Frankfurt (AWS)
2025-11-05T12:34:10.0107631Z • Total duration: 710 ms - View test run details:
2025-11-05T12:34:10.0108906Z ⎋ https://app.datadoghq.eu/synthetics/details/yy7-df3-ncx?resultId=5872150210184434739&batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:34:10.0109977Z ✓ undefined test
2025-11-05T12:34:10.0110189Z
2025-11-05T12:34:10.0110661Z - Waiting for 5 tests (ynm-97u-azm, mqj-rrp-26u, tvp-p4z-2gc, v34-esy-zjx, mby-vm7-emh)…
2025-11-05T12:34:10.0111232Z
2025-11-05T12:41:19.3634375Z ✓ [v34-esy-zjx] [BUY] Complete member checkout flow with registration [Development] - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:41:19.3635717Z • Total duration: 392406 ms - View test run details:
2025-11-05T12:41:19.3637010Z ⎋ https://app.datadoghq.eu/synthetics/details/v34-esy-zjx/result/7462487755695851082?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:41:19.3638679Z
2025-11-05T12:41:19.3639145Z - Waiting for 4 tests (ynm-97u-azm, mqj-rrp-26u, tvp-p4z-2gc, mby-vm7-emh)…
2025-11-05T12:41:19.3639600Z
2025-11-05T12:41:30.1019176Z ✓ [ynm-97u-azm] Gift card store APIs integration test [development] (Terraform generated) - location: Frankfurt (AWS)
2025-11-05T12:41:30.1038233Z • Total duration: 5171 ms - View test run details:
2025-11-05T12:41:30.1043903Z ⎋ https://app.datadoghq.eu/synthetics/details/ynm-97u-azm?resultId=6714997651571466788&batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:41:30.1048493Z ✓ undefined test
2025-11-05T12:41:30.1048745Z
2025-11-05T12:41:30.1049104Z - Waiting for 3 tests (mqj-rrp-26u, tvp-p4z-2gc, mby-vm7-emh)…
2025-11-05T12:41:30.1049554Z
2025-11-05T12:42:55.4121574Z ✓ [mqj-rrp-26u] Test on wrong example.com/api/mobileapp/puc_de/login - location: Frankfurt (AWS)
2025-11-05T12:42:55.4123036Z • Total duration: 357 ms - View test run details:
2025-11-05T12:42:55.4124335Z ⎋ https://app.datadoghq.eu/synthetics/details/mqj-rrp-26u?resultId=984328905990691054&batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:42:55.4125688Z ✓ POST - https://example.com/api/mobileapp/puc_de/login
2025-11-05T12:42:55.4126202Z
2025-11-05T12:42:55.4127006Z ✓ [mby-vm7-emh] [Quality Gate - Development] PUC_NL Integration Path (cloned) - location: Frankfurt (AWS) - device: firefox.laptop_large
2025-11-05T12:42:55.4128823Z • Total duration: 73820 ms - View test run details:
2025-11-05T12:42:55.4130197Z ⎋ https://app.datadoghq.eu/synthetics/details/mby-vm7-emh/result/4617247258016441516?batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:42:55.4131106Z
2025-11-05T12:42:55.4134673Z - Waiting for 1 test (tvp-p4z-2gc)…
2025-11-05T12:42:55.4135019Z
2025-11-05T12:43:00.8488762Z ✓ [tvp-p4z-2gc] Test register on development - location: Frankfurt (AWS)
2025-11-05T12:43:00.8504648Z • Total duration: 884 ms - View test run details:
2025-11-05T12:43:00.8506008Z ⎋ https://app.datadoghq.eu/synthetics/details/tvp-p4z-2gc?resultId=5334114664365413516&batch_id=d22af079-0d54-415a-ba74-6d973c2d04c6&from_ci=true
2025-11-05T12:43:00.8507544Z ✓ POST - https://example.com/api/mobileapp/puc_de/register
2025-11-05T12:43:00.8508092Z
2025-11-05T12:43:01.0323456Z
2025-11-05T12:43:01.0324827Z === REPORT ===
2025-11-05T12:43:01.0329458Z Took 1722380ms
2025-11-05T12:43:01.0329951Z
2025-11-05T12:43:01.0330953Z View full summary in Datadog: https://app.datadoghq.eu/synthetics/explorer/ci?batchResultId=d22af079-0d54-415a-ba74-6d973c2d04c6
2025-11-05T12:43:01.0331943Z
2025-11-05T12:43:01.0332231Z Continuous Testing Summary:
2025-11-05T12:43:01.0333112Z • Test Results: 12 passed, 0 failed
2025-11-05T12:43:01.0334893Z • Max parallelization configured: 1 test running at the same time
2025-11-05T12:43:01.0335603Z • Total Duration: 28m 42s
2025-11-05T12:43:01.0335853Z
2025-11-05T12:43:01.0337494Z View test runs in Test Optimization: https://app.datadoghq.eu/ci/test-runs?query=%40test.framework%3Asynthetics%20%40ci.job.url%3A%22https%3A%2F%2Fgithub.com%2Ffadi-nextgen-shop%2Fngshop-schneiderflow%2Fcommit%2F92c260233eab376408b0e3d6496759185b9d0e0e%2Fchecks%22
2025-11-05T12:43:01.0339112Z
2025-11-05T12:43:01.0339794Z Increase your parallelization to reduce the CI batch duration: https://app.datadoghq.eu/synthetics/settings/continuous-testing
2025-11-05T12:43:01.0401996Z Datadog Synthetics tests succeeded: criticalErrors: 0, passed: 12, previouslyPassed: 0, failedNonBlocking: 0, failed: 0, skipped: 0, notFound: 0, timedOut: 0
2025-11-05T12:43:01.0415596Z Batch URL: https://app.datadoghq.eu/synthetics/explorer/ci?batchResultId=d22af079-0d54-415a-ba74-6d973c2d04c6
2025-11-05T12:43:01.0760685Z ##[group]Run echo 'Batch URL: https://app.datadoghq.eu/synthetics/explorer/ci?batchResultId=d22af079-0d54-415a-ba74-6d973c2d04c6'
2025-11-05T12:43:01.0761856Z [36;1mecho 'Batch URL: https://app.datadoghq.eu/synthetics/explorer/ci?batchResultId=d22af079-0d54-415a-ba74-6d973c2d04c6' [0m
2025-11-05T12:43:01.0762383Z [36;1mecho 'Raw Results:' [0m
2025-11-05T12:43:01.0762613Z [36;1mecho $RAW_RESULTS | jq '.'[0m
2025-11-05T12:43:01.0801119Z shell: /usr/bin/bash -e {0}
2025-11-05T12:43:01.0801379Z env:
2025-11-05T12:43:01.1696680Z RAW_RESULTS: [{"device":{"id":"firefox.laptop_large","name":"Laptop Large","resolution":{"width":1440,"height":1100},"browser":{"version":"143.0.4","user_agent":"Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/143.0.4 DatadogSynthetics","type":"firefox"},"type":"laptop"},"duration":75733,"executionRule":"blocking","isNonFinal":false,"location":"Frankfurt (AWS)","passed":true,"result":{"triggered_at":1762344860130,"time_to_interactive":1306,"status":"passed","steps":[{"status":"passed","vitals_metrics":[],"duration":2087,"warnings":[],"rum_context":{"application_id":"9c8ea197-1522-4b...]
2025-11-05T12:43:01.2601041Z ##[endgroup]
2025-11-05T12:43:01.2782424Z ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/runner/work/ngshop-schneiderflow/ngshop-schneiderflow'. Argument list too long
2025-11-05T12:43:01.2846376Z Cleaning up orphan processes
I had to redact the full raw result because it contains sensitive data. The string is 313145 characters long.
