act icon indicating copy to clipboard operation
act copied to clipboard
verified publisher icon nektos

Frozen local run hanging and not showing up on docker ps

Open misha-sentry opened this issue 3 years ago • 3 comments

Bug report info

➜  security-panther-analysis git:(master) act --bug-report
act version:            0.2.31
GOOS:                   darwin
GOARCH:                 arm64
NumCPU:                 10
Docker host:            DOCKER_HOST environment variable is unset/empty.
Sockets found:
	/var/run/docker.sock
Config files:           
	/Users/mishayalavarthy/.actrc:
		-P ubuntu-latest=catthehacker/ubuntu:full-latest
		-P ubuntu-latest=catthehacker/ubuntu:full-20.04
		-P ubuntu-18.04=catthehacker/ubuntu:full-18.04
Build info:
	Go version:            go1.19
	Module path:           command-line-arguments
	Main version:          
	Main path:             
	Main checksum:         
	Build settings:
		-compiler:            gc
		-ldflags:             -X main.version=0.2.31
		CGO_ENABLED:          1
		CGO_CFLAGS:           
		CGO_CPPFLAGS:         
		CGO_CXXFLAGS:         
		CGO_LDFLAGS:          
		GOARCH:               arm64
		GOOS:                 darwin
Docker Engine:
	Engine version:        20.10.17
	Engine runtime:        runc
	Cgroup version:        2
	Cgroup driver:         cgroupfs
	Storage driver:        overlay2
	Registry URI:          https://index.docker.io/v1/
	OS:                    Docker Desktop
	OS type:               linux
	OS version:            
	OS arch:               aarch64
	OS kernel:             5.10.124-linuxkit
	OS CPU:                5
	OS memory:             7851 MB
	Security options:
		name=seccomp,profile=default
		name=cgroupns

Command used with act

act pull_request --workflows .github/workflows/panther-workflow.yml --container-architecture linux/amd64

act pull_request --workflows panther-workflow.yml --container-architecture linux/amd64 -s PANTHER_API_KEY=[secret] -s PANTHER_API_HOST=[api_host_url]

Describe issue

I see in the bug report that act is having trouble with Docker: Docker host: DOCKER_HOST environment variable is unset/empty.

However, I'm unsure of how to fix this. And if there might be something else contributing to this. I tried adding the act -s for the env vars/secrets, but that triggered act to run and then it would hang again.

I'm also not wanting to run the "sync_prs_from_community_to_dogfood" job.

Link to GitHub repository

No response

Workflow content

# Panther is a Cloud-Native SIEM for the Modern Security Team.
# Copyright (C) 2020 Panther Labs Inc
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see <https://www.gnu.org/licenses/>.

on:
  pull_request:
    branches:
      - master
    types:
      - opened
      - closed
  push:
    branches:
      - master

jobs:
  download_pantherlog_tool:
    name: Download the latest version of pantherlog tool
    runs-on: ubuntu-latest
    steps:
      - name: Download pantherlog & unzip 
        run: curl -sSO "https://panther-community-us-east-1.s3.amazonaws.com/v1.32.4/tools/linux-amd64-pantherlog.zip" && unzip linux-amd64-pantherlog.zip
      - name: Create a pantherlog artifact
        uses: actions/upload-artifact@v3
        with:
          name: pantherlog
          path: pantherlog
          retention-days: 1
  run_unit_tests_using_panther_analysis_tool:
    name: Runs unit tests on detections with panther_analysis_tool
    runs-on: ubuntu-latest
    needs: [download_pantherlog_tool]
    steps:
      - name: Check out the repo
        uses: actions/checkout@v2
      - name: Download the panther_analysis_tool
        run: pip3 install panther_analysis_tool
      - name: Download policyuniverse
        run: pip3 install policyuniverse
      - name: Run unit tests within the Detections folder
        run: |
          for dir in rules/*; do
            if [[ "$dir" =~ .*_rules.* ]]; then
              panther_analysis_tool test
            fi
          done
  upload_detections_using_panther_analysis_tool:
    name: Upload detections to the panther console using panther_analysis_tool
    runs-on: ubuntu-latest
    needs: [download_pantherlog_tool, run_unit_tests_using_panther_analysis_tool]
    env: |
      PANTHER_API_KEY:  ${{ secrets.PANTHER_SECRET }}
      PANTHER_API_HOST: "https://api.sentry.runpanther.net/public/graphql"
    steps:
      - name: Checkout the repo
        uses: actions/checkout@v2
      - name: Download panther_analysis_tool
        run: pip3 install panther_analysis_tool
      - name: Download policyuniverse
        run: pip3 install policyuniverse
      - name: Loop through folders ending in _rules and upload to our Panther instance 
        run: |
          for dir in rules/*; do
            if [[ "$dir" =~ .*_rules.* ]]; then
              panther_analysis_tool upload --path "$dir" --skip-tests --api-token $PANTHER_API_KEY --api-host $PANTHER_API_HOST 
            fi
          done

Relevant log output

WARN  ⚠ You are using Apple M1 chip and you have not specified container architecture, you might encounter issues while running act. If so, try running it with '--container-architecture linux/amd64'. ⚠  
DEBU[0000] Loading environment from /Users/mishayalavarthy/github/security-panther-analysis/.env 
DEBU[0000] Loading secrets from /Users/mishayalavarthy/github/security-panther-analysis/.secrets 
DEBU[0000] Loading workflows from '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows' 
DEBU[0000] Loading workflows recursively                
DEBU[0000] Found workflow 'asana.yml' in '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/asana.yml' 
DEBU[0000] Found workflow 'lint-test.yml' in '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/lint-test.yml' 
DEBU[0000] Found workflow 'panther-workflow.yml' in '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/panther-workflow.yml' 
DEBU[0000] Reading workflow '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/asana.yml' 
DEBU[0000] Reading workflow '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/lint-test.yml' 
DEBU[0000] Reading workflow '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/panther-workflow.yml' 
DEBU[0000] Planning event: push                         
DEBU[0000] Loading slug from git directory '/Users/mishayalavarthy/github/security-panther-analysis/.git' 
DEBU[0000] Loading revision from git directory '/Users/mishayalavarthy/github/security-panther-analysis/.git' 
DEBU[0000] Found revision: 9f06f00a7053eeb54865bf3d5aab785814be334b 
DEBU[0000] HEAD points to '9f06f00a7053eeb54865bf3d5aab785814be334b' 
DEBU[0000] using github ref: refs/heads/master          
DEBU[0000] Found revision: 9f06f00a7053eeb54865bf3d5aab785814be334b 
DEBU[0000] expression '${{ format('sync-{0}', github.event.pull_request.head.ref) }}' rewritten to 'format('{0}', format('sync-{0}', github.event.pull_request.head.ref))' 
DEBU[0000] evaluating expression 'format('{0}', format('sync-{0}', github.event.pull_request.head.ref))' 
DEBU[0000] expression 'format('{0}', format('sync-{0}', github.event.pull_request.head.ref))' evaluated to '%!t(string=sync-)' 
DEBU[0000] Loading slug from git directory '/Users/mishayalavarthy/github/security-panther-analysis/.git' 
DEBU[0000] Loading revision from git directory '/Users/mishayalavarthy/github/security-panther-analysis/.git' 
DEBU[0000] Found revision: 9f06f00a7053eeb54865bf3d5aab785814be334b 
DEBU[0000] HEAD points to '9f06f00a7053eeb54865bf3d5aab785814be334b' 
DEBU[0000] using github ref: refs/heads/master          
DEBU[0000] Found revision: 9f06f00a7053eeb54865bf3d5aab785814be334b 
[panther-workflow.yml/Syncs merged PRs from Panther-Analysis Community to Panther-Analysis-Dogfood] [DEBUG] evaluating expression 'github.repository_owner == 'panther-labs' && 
( github.event.pull_request.merged == true && !contains(github.event.pull_request.labels.*.name, 'dogfood-backport') )
'
[panther-workflow.yml/Syncs merged PRs from Panther-Analysis Community to Panther-Analysis-Dogfood] [DEBUG] expression 'github.repository_owner == 'panther-labs' && 
( github.event.pull_request.merged == true && !contains(github.event.pull_request.labels.*.name, 'dogfood-backport') )
' evaluated to 'false'
[panther-workflow.yml/Syncs merged PRs from Panther-Analysis Community to Panther-Analysis-Dogfood] [DEBUG] Skipping job 'Syncs merged PRs from Panther-Analysis Community to Panther-Analysis-Dogfood' due to 'github.repository_owner == 'panther-labs' && 
( github.event.pull_request.merged == true && !contains(github.event.pull_request.labels.*.name, 'dogfood-backport') )
'
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] evaluating expression 'success()'
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] expression 'success()' evaluated to 'true'
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] 🚀  Start image=catthehacker/ubuntu:full-20.04
[panther-workflow.yml/Download the latest version of pantherlog tool                              ]   🐳  docker pull image=catthehacker/ubuntu:full-20.04 platform= username= forcePull=false
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG]   🐳  docker pull catthehacker/ubuntu:full-20.04
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Image exists? false
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] pulling image 'docker.io/catthehacker/ubuntu:full-20.04' ()
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling from catthehacker/ubuntu :: full-20.04
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: 16ec32c2132b
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: e844d0dd6bf6
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: 1e6e2a2dd736
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: ebaacada9b34
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: 47b6cb30d160
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: e797c87a8318
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: bb8d9f7a92ff
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: 1a426ad63194
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Pulling fs layer :: 9aa1f02e7614
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: ebaacada9b34
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: 47b6cb30d160
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: e797c87a8318
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: bb8d9f7a92ff
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: 1a426ad63194
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Waiting :: 9aa1f02e7614
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 1e6e2a2dd736 :: [>                                                  ]  37.51kB/3.624MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 1e6e2a2dd736 :: [=============================>                     ]  2.136MB/3.624MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: e844d0dd6bf6 :: [>                                                  ]  81.73kB/7.77MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 16ec32c2132b :: [>                                                  ]  301.4kB/28.57MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 1e6e2a2dd736 :: [=================================================> ]   3.59MB/3.624MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Download complete :: 1e6e2a2dd736
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 16ec32c2132b :: [=>                                                 ]  891.2kB/28.57MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: e844d0dd6bf6 :: [===>                                               ]  578.5kB/7.77MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 16ec32c2132b :: [======>                                            ]  3.537MB/28.57MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: e844d0dd6bf6 :: [=================>                                 ]  2.782MB/7.77MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: e844d0dd6bf6 :: [========================>                          ]  3.855MB/7.77MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 16ec32c2132b :: [=========>                                         ]   5.29MB/28.57MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: e844d0dd6bf6 :: [============================================>      ]  6.989MB/7.77MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Downloading :: 16ec32c2132b :: [==============>                                    ]   8.53MB/28.57MB
[panther-workflow.yml/Download the latest version of pantherlog tool                              ] [DEBUG] Download complete :: e844d0dd6bf6



^ This continues as it attempts to download and extract.

Additional information

No response

misha-sentry avatar Sep 12 '22 18:09 misha-sentry

New error: ➜ workflows git:(master) ✗ act pull_request --workflows panther-workflow.yml --container-architecture linux/amd64 -s PANTHER_API_KEY=[key] -s PANTHER_API_HOST=host] -s GITHUB_TOKEN=[key] [panther-workflow.yml/Download the latest version of pantherlog tool] 🚀 Start image=catthehacker/ubuntu:full-20.04 [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker pull image=catthehacker/ubuntu:full-20.04 platform=linux/amd64 username= forcePull=false [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker create image=catthehacker/ubuntu:full-20.04 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[] [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker run image=catthehacker/ubuntu:full-20.04 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[] [panther-workflow.yml/Download the latest version of pantherlog tool] ☁ git clone 'https://github.com/actions/upload-artifact' # ref=v3 [panther-workflow.yml/Download the latest version of pantherlog tool] ⭐ Run Main Download pantherlog & unzip [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/0] user= workdir= | Archive: linux-amd64-pantherlog.zip | inflating: pantherlog
[panther-workflow.yml/Download the latest version of pantherlog tool] ✅ Success - Main Download pantherlog & unzip [panther-workflow.yml/Download the latest version of pantherlog tool] ⭐ Run Main Create a pantherlog artifact [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker cp src=/Users/mishayalavarthy/.cache/act/actions-upload-artifact@v3/ dst=/var/run/act/actions/actions-upload-artifact@v3/ [panther-workflow.yml/Download the latest version of pantherlog tool] 🐳 docker exec cmd=[node /var/run/act/actions/actions-upload-artifact@v3/dist/index.js] user= workdir= [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::followSymbolicLinks 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::implicitDescendants 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::omitBrokenSymbolicLinks 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::followSymbolicLinks 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::implicitDescendants 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::omitBrokenSymbolicLinks 'true' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::Search path '/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/pantherlog' [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::File:/Users/mishayalavarthy/github/security-panther-analysis/.github/workflows/pantherlog was found using the provided searchPath | With the provided path, there will be 1 file uploaded [panther-workflow.yml/Download the latest version of pantherlog tool] 💬 ::debug::Root artifact directory is /Users/mishayalavarthy/github/security-panther-analysis/.github/workflows | Starting artifact upload | For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging | Artifact name is valid! [panther-workflow.yml/Download the latest version of pantherlog tool] ❗ ::error::Unable to get ACTIONS_RUNTIME_TOKEN env variable [panther-workflow.yml/Download the latest version of pantherlog tool] ❌ Failure - Main Create a pantherlog artifact [panther-workflow.yml/Download the latest version of pantherlog tool] exitcode '1': failure [panther-workflow.yml/Download the latest version of pantherlog tool] 🏁 Job failed Error: Job 'Download the latest version of pantherlog tool' failed

misha-sentry avatar Sep 12 '22 19:09 misha-sentry

For your second error you have to add this flag to the act cli: --artifact-server-path <path/to/store/artifacts>.

Don't ask me why this flag is needed, I prefer that act would ask for it after you uploaded artifact if you omitted the path

ChristopherHX avatar Sep 12 '22 19:09 ChristopherHX

Thanks buddy. Great Works!

Cori1109 avatar Sep 19 '22 07:09 Cori1109

Issue is stale and will be closed in 14 days unless there is new activity

github-actions[bot] avatar Oct 20 '22 00:10 github-actions[bot]