bug: Localstack Managed Pods - Pull response 404

[Tobolov]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

It seems that the managed backend for pods is currently broken. When going to pull a cloudpod using the managed backend, I received the below exception.

C:\Users\mtobolov>python C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\\localstack pod pull manualworkingtest
Pulling state of cloud pod "manualworkingtest" from managed storage
Traceback (most recent call last):
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\localstack", line 23, in <module>
    main()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\localstack", line 19, in main
    main.main()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\cli\main.py", line 6, in main
    cli()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\cli\plugin.py", line 30, in __call__
    self.group(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\cli\localstack.py", line 107, in cmd_pod_pull
    pods_client.pull_state(name,asevx)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\bootstrap\pods_client.py", line 333, in pull_state
    backend.pull()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\bootstrap\pods_client.py", line 151, in pull
    download(presigned_url,zip_path)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\utils\common.py", line 972, in download
    raise Exception("Failed to download %s, response code %s" % (url, r.status_code))
Exception: Failed to download https://localstack-pods-a2bac14f.s3.amazonaws.com/pod-5efd7ab7-10.zip?AWSAccessKeyId=<removed>&Signature=A4Wc%2Fhc9ZbGNTwDsxlup0Vc%2FE64%3D&x-amz-security-token=<removed>&Expires=1634598784, response code 404

Expected Behavior

The pod is pulled successfully and the command exits with status code 0 gracefully.

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

docker-compose.yml:

version: '2.1'

services:
  localstack_ltest:
    container_name: "ltest"
    image: localstack/localstack:0.12.18
    ports:
      - "4566:4566"
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LOCALSTACK_API_KEY=<removed>
      - LAMBDA_EXECUTOR=docker-reuse
      - DATA_DIR=/tmp/localstack-data
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "C://tmp2/localstack/:/tmp/localstack/"
    restart: always

1. Run docker-compose up -d against the docker compose file above
2. Run$env:DATA_DIR="C://tmp2/localstack" and $env:LOCALSTACK_API_KEY="<removed>"
3. Run localstack login and enter credentials
4. Run localstack pod pull test, where test is the name of a managed cloudpod which has been pushed to.

Environment

- OS: Windows 10
- LocalStack Container: 
LocalStack version: 0.12.18
LocalStack Docker container id: 64395b488268
LocalStack build date: 2021-09-23
LocalStack build git hash: 2c436bad

- Localstack Pip Packages:
localstack                0.12.17.5
localstack-client         1.26
localstack-ext            0.12.17.9

Anything else?

No response

[whummer]

Hi @Tobolov , apologies for the delay on this one. Can you please confirm if this issue still persists after pulling the latest Docker image?

Please note that this feature is under very active development - we've recently done quite a few changes and refactorings around cloud pods. We'll release a new, completely revised version over the next couple of days. Please keep us posted how it goes - thanks for your help! 🙏

[Tobolov]

Hi @whummer, ~Can confirm that I am now able to pull managed pods!~

I did get the the following error running localstack pod pull test though. Just had to install dill via pip:

ModuleNotFoundError: No module named 'dill'

Localstack pack versions: localstack 0.12.20 localstack-client 1.26 localstack-ext 0.12.19.25 localstack-plugin-loader 1.1.1

[Tobolov]

Was trying again to test something else and it didn't work. Steps to reproduce: docker-compose.yml:

version: '2.1'
services:
  localstack_ltest:
    container_name: "ltest"
    image: localstack/localstack:0.12.18
    ports:
      - "4566:4566"
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LOCALSTACK_API_KEY=<removed>
      - LAMBDA_EXECUTOR=docker-reuse
      - DATA_DIR=/tmp/localstack-data
      - LOCALSTACK_API_KEY=<removed>
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "C://tmp2/localstack/:/tmp/localstack/"
    restart: always

1. Create new managed pog testv2
2. $env:DATA_DIR="C://tmp2/localstack"
3. $env:LOCALSTACK_API_KEY=""
4. localstack login
5. docker-compose up -d
6. aws apigatewayv2 create-api --protocol-type HTTP --name test-api --endpoint-url http://127.0.0.1:4566
7. aws s3 mb s3://test-bk --endpoint-url http://127.0.0.1:4566
8. localstack pod push testv2
9. docker-compose down
10. Delete the directory C://tmp2/localstack/
11. docker-compose up -d
12. localstack pod pull testv2
13. Fails with:

C:\Users\mtobolov\Desktop>python C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\\localstack pod pull testv2
Pulling state of cloud pod "testv2" from managed storage
Traceback (most recent call last):
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\localstack", line 23, in <module>
    main()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\Scripts\localstack", line 19, in main
    main.main()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\cli\main.py", line 6, in main
    cli()
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\cli\plugin.py", line 15, in __call__
    self.group(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\mtobolov\AppData\Roaming\Python\Python39\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\cli\localstack.py", line 279, in cmd_pod_pull
    pods_client.pull_state(name)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\bootstrap\pods_client.py", line 697, in pull_state
    backend.pull(inject_version_state=inject_version_state,reset_state_before=reset_state_before)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack_ext\bootstrap\pods_client.py", line 455, in pull
    download(presigned_url,zip_path)
  File "C:\Users\mtobolov\AppData\Local\Programs\Python\Python39\lib\site-packages\localstack\utils\common.py", line 1103, in download
    raise Exception("Failed to download %s, response code %s" % (url, r.status_code))
Exception: Failed to download https://localstack-pods-a2bac14f.s3.amazonaws.com/pod-e5c4fbc3-2.zip?AWSAccessKeyId=<removed>&Signature=zedFvqEMNH512YLTubZxd7BbbPc%3D&x-amz-security-token=<removed>&Expires=1636436504, response code 404

Container version: LocalStack version: 0.12.18 LocalStack Docker container id: 18724d523f85 LocalStack build date: 2021-09-23 LocalStack build git hash: 2c436bad

Pip version: localstack 0.12.20 localstack-client 1.26 localstack-ext 0.12.19.25 localstack-plugin-loader 1.1.1

[giograno]

Hi @Tobolov, with our recent v.1.0.0 we completely revamped the Cloud Pods feature. Would you pull the latest Docker image and give it another go? The following sequence of commands should now work.

aws apigatewayv2 create-api --protocol-type HTTP --name test-api --endpoint-url http://127.0.0.1:4566/
aws s3 mb s3://test-bk --endpoint-url http://127.0.0.1:4566/
localstack pod push --name testv2 
# the command below will only create the pod locally
# localstack pod push --name testv2 --local

# restart
localstack pod pull --name testv2 
# the command below will work with local pods instead
# localstack pod inject --name testv2 

Please feel free to have a look at our documentation here. Keep us posted 🙏