cli icon indicating copy to clipboard operation
cli copied to clipboard

Container creation error: You should always set the Memory limit when using Memoryswap limit

Open samikrc opened this issue 1 year ago • 5 comments

I am trying to run the fn java example on WSL2 (Ubuntu 22.04) with docker engine, and getting the following error when I try to invoke the function.

The client shows:

$ fn invoke java-app javafn
Error invoking function. status: 500 message: internal server error

The server debug log shows:

time="2024-12-12T06:34:20Z" level=debug msg="docker inspect image" app_id=01JEVG6929NG8G00GZJ0000003 call_id=01JEWS6MMKNG8G00GZJ0000002 container_id=01JEWS6MMKNG8G00GZJ0000002 cpus= fn_id=01JEVGHBEVNG8G00GZJ0000004 idle_timeout=30 image="fn_local_registry/javafn:0.0.2" memory=1024 stack=ValidateImage
time="2024-12-12T06:34:20Z" level=debug msg="docker create container" app_id=01JEVG6929NG8G00GZJ0000003 call_id=01JEWS6MMKNG8G00GZJ0000002 container_id=01JEWS6MMKNG8G00GZJ0000002 cpus= fn_id=01JEVGHBEVNG8G00GZJ0000004 idle_timeout=30 image="fn_local_registry/javafn:0.0.2" memory=1024 stack=CreateContainer
time="2024-12-12T06:34:20Z" level=error msg="Could not create container" app_id=01JEVG6929NG8G00GZJ0000003 container_id=01JEWS6MMKNG8G00GZJ0000002 cpus= error="API error (400): You should always set the Memory limit when using Memoryswap limit, see usage" fn_id=01JEVGHBEVNG8G00GZJ0000004 idle_timeout=30 image="fn_local_registry/javafn:0.0.2" memory=1024 stack=CreateContainer
time="2024-12-12T06:34:20Z" level=info msg="hot function failure" app_id=01JEVG6929NG8G00GZJ0000003 container_id=01JEWS6MMKNG8G00GZJ0000002 cpus= error="API error (400): You should always set the Memory limit when using Memoryswap limit, see usage" fn_id=01JEVGHBEVNG8G00GZJ0000004 idle_timeout=30 image="fn_local_registry/javafn:0.0.2" memory=1024
time="2024-12-12T06:34:20Z" level=info msg="hot function failure" app_id=01JEVG6929NG8G00GZJ0000003 container_id=01JEWS6MMKNG8G00GZJ0000002 cpus= error="Container failed to initialize, please ensure you are using the latest fdk and check the logs" fn_id=01JEVGHBEVNG8G00GZJ0000004 idle_timeout=30 image="fn_local_registry/javafn:0.0.2" memory=1024
time="2024-12-12T06:34:20Z" level=info msg="container wait error, sending error to client" action="server.handleFnInvokeCall)-fm" app_id=01JEVG6929NG8G00GZJ0000003 call_id=01JEWS6MMHNG8G00GZJ0000001 error="API error (400): You should always set the Memory limit when using Memoryswap limit, see usage" fn_id=01JEVGHBEVNG8G00GZJ0000004
time="2024-12-12T06:34:20Z" level=error msg="internal server error" action="server.handleFnInvokeCall)-fm" error="API error (400): You should always set the Memory limit when using Memoryswap limit, see usage" fn_id=01JEVGHBEVNG8G00GZJ0000004 stack="goroutine 27 [running]:
runtime/debug.Stack(0xc4201502a0, 0x10852c0, 0xc4204be460)
        /usr/local/go/src/runtime/debug/stack.go:24 +0xa7
github.com/fnproject/fn/api/server.HandleErrorResponse(0x10902a0, 0xc42041ccf0, 0x7f749e2eed40, 0xc4200ee2c0, 0x10852c0, 0xc4204be460)
        /go/src/github.com/fnproject/fn/api/server/error_response.go:57 +0x57f
github.com/fnproject/fn/api/server.handleErrorResponse(0xc4200ee2c0, 0x10852c0, 0xc4204be460)
        /go/src/github.com/fnproject/fn/api/server/error_response.go:25 +0xa9
github.com/fnproject/fn/api/server.(*Server).handleFnInvokeCall(0xc4204c00e0, 0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/runner_fninvoke.go:53 +0x1da
github.com/fnproject/fn/api/server.(*Server).(github.com/fnproject/fn/api/server.handleFnInvokeCall)-fm(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/server.go:1022 +0x34
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.(*Server).runMiddleware(0xc4204c00e0, 0xc4200ee2c0, 0x0, 0x0, 0x0)
        /go/src/github.com/fnproject/fn/api/server/middleware.go:42 +0x29f
github.com/fnproject/fn/api/server.(*Server).rootMiddlewareWrapper.func1(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/middleware.go:32 +0x52
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.panicWrap(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/gin_middlewares.go:197 +0x51
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.panicWrap(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/gin_middlewares.go:197 +0x51
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.apiMetricsWrap.func1.1(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/gin_middlewares.go:158 +0x316
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.traceWrap(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/gin_middlewares.go:101 +0x5ae
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/api/server.loggerWrap(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/api/server/gin_middlewares.go:214 +0x18e
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/context.go:108 +0x43
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc4204865a0, 0xc4200ee2c0)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/gin.go:361 +0x586
github.com/fnproject/fn/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc4204865a0, 0x108fbe0, 0xc42041a7d0, 0xc420014100)
        /go/src/github.com/fnproject/fn/vendor/github.com/gin-gonic/gin/gin.go:326 +0x153
github.com/fnproject/fn/vendor/go.opencensus.io/plugin/ochttp.(*Handler).ServeHTTP(0xc4202080f0, 0x108fbe0, 0xc42041a7d0, 0xc420152c00)
        /go/src/github.com/fnproject/fn/vendor/go.opencensus.io/plugin/ochttp/server.go:86 +0x1c8
net/http.serverHandler.ServeHTTP(0xc42046e8f0, 0x108ea60, 0xc420010000, 0xc420152c00)
        /usr/local/go/src/net/http/server.go:2697 +0xbc
net/http.(*conn).serve(0xc420438000, 0x10901e0, 0xc4201c65c0)
        /usr/local/go/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2798 +0x27b

I have seen this error around a few places on the net, but have not seen any solution so far. By looking at the error message and the code, the closest I have come to is that the error comes from line# 491 in https://github.com/fnproject/fn/blob/master/api/agent/drivers/docker/cookie.go, in the CreateContainer function. Any pointers on how to solve this?

samikrc avatar Dec 12 '24 10:12 samikrc

This issue appears to occur with Docker engine version 27.x. Version 26.0 did not have the issue.

gontaneko53 avatar Dec 19 '24 09:12 gontaneko53

@gontaneko53 Thanks - I will try downgrading and check again.

Edit: reporting back - downgrading to 26.x.y does seem to work. Might be a docker engine issue then.

samikrc avatar Dec 19 '24 10:12 samikrc

same issue with ruby runtime, I tried docker 28.x, 26.0.2, 26.1.4 and nothing worked, always getting this error message You should always set the Memory limit when using Memoryswap limit

i am on mac m2 and using colima

update: it seems the problem is with colima cause it uses lima and it does not support swap setting, switching to rancher desktop solved the problem with this error

Teckden avatar Feb 20 '25 18:02 Teckden

Same error with WSL (Debian 12.10).

Docker version:

Server: Docker Engine - Community
 Engine:
  Version:          28.0.4
  API version:      1.48 (minimum version 1.24)
  Go version:       go1.23.7
  Git commit:       6430e49
  Built:            Tue Mar 25 15:07:22 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.26
  GitCommit:        753481ec61c7c8955a23d6ff7bc8e4daed455734
 runc:
  Version:          1.2.5
  GitCommit:        v1.2.5-0-g59923ef
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

cvisinoni avatar Apr 07 '25 12:04 cvisinoni

https://github.com/fnproject/fn/pull/1611 - This might solve the issue. At least it solved it for me.

NachiMK avatar Apr 25 '25 02:04 NachiMK