WARNING! Function zip file is 3MB.The maximum size allowed is 1MB: please use package.exclude directives to include only the required files

[engmsaleh]

Is this a BUG REPORT or FEATURE REQUEST?: Help request

What happened: I'm trying to use Kubeless with serverless and I have several functions inside my serverless.yml file and I have separate dependencies for each function. When I'm trying to deploy it using serverless-python-requirements I got the following error

Serverless: Load command config:credentials
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command requirements
Serverless: Load command requirements:clean
Serverless: Load command requirements:install
Serverless: Load command requirements:cleanCache
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Generated requirements from /Users/myuser/Repos/project-name/qwallet-kubeless/helpers/local_banks/requirements.txt in /Users/myuser/Repos/project-name/qwallet-kubeless/helpers/.serverless/local_banks/requirements.txt...
Serverless: Installing requirements from /Users/myuser/Repos/project-name/qwallet-kubeless/helpers/.serverless/local_banks/requirements/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.7
Serverless: Running docker run --rm -v /Users/myuser/Repos/project-name/qwallet-kubeless/helpers/.serverless/local_banks/requirements\:/var/task\:z -u 0 lambci/lambda\:build-python3.7 /bin/sh -c 'python3.7 -m pip install -t /var/task/ -r /var/task/requirements.txt && find /var/task -name \\*.so -exec strip \\{\\} \\;'...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Injecting required Python packages to package...
Serverless: WARNING! Function zip file is 3MB. The maximum size allowed is 1MB: please use package.exclude directives to include only the required files
Serverless: Unable to find any running pod for get-local-banks. Retrying...
Serverless: Unable to find any running pod for get-local-banks. Retrying...
Serverless: Unable to find any running pod for get-local-banks. Retrying...
Serverless: Unable to find any running pod for get-local-banks. Retrying...
Serverless: Giving up, unable to retrieve the status of the get-local-banks deployment.
/Users/myuser/Repos/project-name/qwallet-kubeless/helpers/node_modules/bluebird/js/release/async.js:61
        fn = function () { throw arg; };
                           ^

Error: Unable to retrieve the status of the get-local-banks deployment
    at module.exports.logError (/usr/local/lib/node_modules/serverless/lib/classes/Error.js:92:11)
    at process.on (/usr/local/lib/node_modules/serverless/bin/serverless:21:3)
    at process.emit (events.js:182:13)
    at /Users/mohamedsaleh/Repos/qwallet/qwallet-kubeless/helpers/node_modules/bluebird/js/release/debuggability.js:199:33
    at activeFireEvent (/Users/mohamedsaleh/Repos/qwallet/qwallet-kubeless/helpers/node_modules/bluebird/js/release/debuggability.js:242:44)
    at fireRejectionEvent (/Users/mohamedsaleh/Repos/qwallet/qwallet-kubeless/helpers/node_modules/bluebird/js/release/debuggability.js:632:14)
    at Promise._notifyUnhandledRejection (/Users/mohamedsaleh/Repos/qwallet/qwallet-kubeless/helpers/node_modules/bluebird/js/release/debuggability.js:65:9)
    at Timeout._onTimeout (/Users/mohamedsaleh/Repos/qwallet/qwallet-kubeless/helpers/node_modules/bluebird/js/release/debuggability.js:44:14)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)

My serverless file looks like this

provider:
  name: kubeless
  runtime: python3.7
  stage: ${opt:stage, 'dev'} #TODO: Check if could switch cluster automatically with stage
  hostname: ${self:custom.hostname.${self:provider.stage}}
  namespace: lambda
  ingress:
    additionalAnnotations:
      kubernetes.io/ingress.class: "kong"


# Custom Variables
custom:
  stages: # for stage manager
    - dev
    - stage
    - prod
  hostname:
    dev: api.dev.hostname.com
    stage: api.stage.hostname.com
    prod:
  pythonRequirements: #This is used to package dependancy with fucntions 
    dockerizePip: true
    # dockerImage: lambci/lambda:build-python3.7
    dockerFile: Dockerfile
    slim: true

package:
  individually: true

functions:
  # get-faqs:
  #   name: ${self:service}-getFAQs
  #   handler: get_faqs.list
  #   module: faqs
  #   timeout: 10
  #   events:
  #     - http:
  #         path: /faqs
  #         method: get

  get-local-banks:
    name: ${self:service}-${self:provider.stage}-getLocalBanks
    handler: get_local_banks.list
    module: local_banks
    timeout: 10
    events:
      - http:
          path: /local_banks
          method: get
    # pythonRequirements:
    #   dockerizePip: true
    #   slim: true
    #   dockerFile: Dockerfile

plugins:
  - serverless-kubeless
  - serverless-stage-manager
  - serverless-python-requirements

What you expected to happen: deploy function successfully without any problems

How to reproduce it (as minimally and precisely as possible): I have a folder for each function and inside each function, I have its requirements.txt

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-19T22:12:47Z", GoVersion:"go1.12.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.9", GitCommit:"16236ce91790d4c75b79f6ce96841db1c843e7d2", GitTreeState:"clean", BuildDate:"2019-03-25T06:30:48Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

• Kubeless version (use kubeless version): Kubeless version: v1.0.3

• Cloud provider or physical cluster: AWS, but private cluster using Kops

[andresmgot]

hi @engmsaleh,

The directory containing your serverless.yaml file should be under 1MB in order to be deployed as it is. Maybe you have in that directory installed dependencies? Note that you don't need to have the dependencies downloaded in your local directory, they will be installed on the fly when deploying a function.

[engmsaleh]

Hi @andresmgot I have tried the following and failed:

• I have tried functions and not modules like the following

  get-local-banks:
    name: ${self:service}-${self:provider.stage}-getLocalBanks
    handler: get_local_banks.list
    # module: local_banks
    timeout: 10
    events:
      - http:
          path: /local_banks
          method: get
    environment:
      DB_HOST: ${self:custom.dbHost.${self:provider.stage}}
      DB_NAME: ${self:custom.dbName.${self:provider.stage}}
      DB_USER: ${self:custom.dbUser.${self:provider.stage}}
      DB_PASSWORD: ${self:custom.dbPassword.${self:provider.stage}}

This function for example needs the psycopg2 But this approach gives the following

Serverless: Pods status: {"waiting":{"reason":"PodInitializing"}}
Serverless: Pods status: {"terminated":{"exitCode":1,"reason":"Error","startedAt":"2019-05-23T21:25:02Z","finishedAt":"2019-05-23T21:25:02Z","containerID":"docker://2830624bbac1fa70028d3f31a28c900d4a45778e65e7b32d06564d0b03e80c26"}}
Serverless: Pods status: {"waiting":{"reason":"CrashLoopBackOff","message":"Back-off 10s restarting failed container=get-local-banks pod=get-local-banks-dfdb9496d-w22wh_lambda(289d7b2b-7da1-11e9-8089-02d40275f760)"}}
Serverless: Pods status: {"terminated":{"exitCode":1,"reason":"Error","startedAt":"2019-05-23T21:25:15Z","finishedAt":"2019-05-23T21:25:15Z","containerID":"docker://2740e35ef10362702bddc21ea0aca90854b410936cbc6039bc97957055b82674"}}
Serverless: Pods status: {"waiting":{"reason":"CrashLoopBackOff","message":"Back-off 20s restarting failed container=get-local-banks pod=get-local-banks-dfdb9496d-w22wh_lambda(289d7b2b-7da1-11e9-8089-02d40275f760)"}}
Serverless: ERROR: Failed to deploy the function
Serverless: Pods status: {"terminated":{"exitCode":1,"reason":"Error","startedAt":"2019-05-23T21:25:41Z","finishedAt":"2019-05-23T21:25:42Z","containerID":"docker://515b492ff7ac9805fe9ca02b8e35ff3c5b17bb6f45bb84389e7a2dfbffdd1406"}}
Serverless: ERROR: Failed to deploy the function

• I have tried the requirements.txt either as a function or as a module and both results on more than 3MB and finally failed to deploy

Please advise if I miss something or there is another approach

they will be installed on the fly when deploying a function.

Could you clarify how this is done