fn icon indicating copy to clipboard operation
fn copied to clipboard

gvisor smoke test failed

Open rdallman opened this issue 5 years ago • 0 comments

posting to help/save anyone from trying this if they search beforehand, though it'd be great if there was an easy fix, this has potential anyway for an easy but solid isolation story at least...

following instructions from https://github.com/google/gvisor#running-with-docker after configuring docker, started up an fn server that uses that docker and used the fdk-go hello world function. The function container starts and the unix socket opens but the unix socket isn't able to be connected to from fn, fn logs are here:

DEBU[2019-02-03T22:04:59-08:00] Hot function launcher starting                action="server.(*Server).handleFnInvokeCall-fm" app_id=01D2RHGQ15180043RZJ0000001 fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000007 launcher_timeou
t=1h0m0s                                                                                                                                                                                                                                      
DEBU[2019-02-03T22:04:59-08:00] setting tmpfs                                 app_id=01D2RHGQ15180043RZJ0000001 call_id=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 image=
"hello:0.0.26" memory=128 options= stack=CreateCookie target=/tmp                                                                                                                                                                             
DEBU[2019-02-03T22:04:59-08:00] setting hostname                              app_id=01D2RHGQ15180043RZJ0000001 call_id=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C hostname=debian id=01D2VHR7A6180043RZJ0000008 idle_t
imeout=30 image="hello:0.0.26" memory=128 stack=CreateCookie                                                                                                                                                                                  
DEBU[2019-02-03T22:04:59-08:00] docker auth image                             app_id=01D2RHGQ15180043RZJ0000001 call_id=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 image=
"hello:0.0.26" memory=128 stack=AuthImage                                                                                                                                                                                                     
DEBU[2019-02-03T22:04:59-08:00] docker inspect image                          app_id=01D2RHGQ15180043RZJ0000001 call_id=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 image=
"hello:0.0.26" memory=128 stack=ValidateImage                                                                                                                                                                                                 
DEBU[2019-02-03T22:04:59-08:00] docker create container                       app_id=01D2RHGQ15180043RZJ0000001 call_id=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 image=
"hello:0.0.26" memory=128 stack=CreateContainer                                                                                                                                                                                               
DEBU[2019-02-03T22:04:59-08:00] Starting container execution                  app_id=01D2RHGQ15180043RZJ0000001 container=01D2VHR7A6180043RZJ0000008 cpus= fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 imag
e="hello:0.0.26" memory=128                                                                                                                                                                                                                   
DEBU[2019-02-03T22:05:00-08:00] fsnotify event                                app_id=01D2RHGQ15180043RZJ0000001 cpus= event="\"/tmp/iofs883530191/phonylsnr.sock\": CREATE" fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idl
e_timeout=30 image="hello:0.0.26" memory=128                                                                                                                                                                                                  
DEBU[2019-02-03T22:05:00-08:00] fsnotify event                                app_id=01D2RHGQ15180043RZJ0000001 cpus= event="\"/tmp/iofs883530191/phonylsnr.sock\": CHMOD" fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle
_timeout=30 image="hello:0.0.26" memory=128                 
DEBU[2019-02-03T22:05:00-08:00] fsnotify event                                app_id=01D2RHGQ15180043RZJ0000001 cpus= event="\"/tmp/iofs883530191/phonylsnr.sock\": CHMOD" fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle
_timeout=30 image="hello:0.0.26" memory=128
DEBU[2019-02-03T22:05:00-08:00] fsnotify event                                app_id=01D2RHGQ15180043RZJ0000001 cpus= event="\"/tmp/iofs883530191/lsnr.sock\": CREATE" fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_tim
eout=30 image="hello:0.0.26" memory=128                                                                                                                                                                                                                                   
ERRO[2019-02-03T22:05:00-08:00] Failed to check socket destination            app_id=01D2RHGQ15180043RZJ0000001 cpus= error="listener file is not a socket" fn_id=01D2SN4A7Z180043RZJ000000C id=01D2VHR7A6180043RZJ0000008 idle_timeout=30 ima
ge="hello:0.0.26" memory=128
ERRO[2019-02-03T22:05:00-08:00] api error                                     action="server.(*Server).handleFnInvokeCall-fm" code=502 error="container failed to initialize, please ensure you are using the latest fdk / format and check th
e logs" fn_id=01D2SN4A7Z180043RZJ000000C

added some prints and it appears that inside the container the file mode is set to have the socket bit set (ie it's not just a gvisor bug in not creating the socket file with that bit set, presumably, though if it's incompatible with the standard linux way of doing this that would explain it), however from the host (fn) it cannot see the socket bit and is unable to connect to the file, all other permissions are preserved as expected, though. it's possible this is an easy fix or impossible due to the gvisor isolation model, some cursory scouring of the internet didn't turn up very much.

rdallman avatar Feb 22 '19 01:02 rdallman