boundary icon indicating copy to clipboard operation
boundary copied to clipboard

Published 20 hours ago • verified publisher icon hashicorp

Kubernetes worker OOMKilled

Open rfc2119 opened this issue 2 months ago • 0 comments

Describe the bug My Kubernetes workers become unresponsive after a while, similar to this issue https://github.com/hashicorp/boundary/issues/2598. Unfortunately, I am unable to figure out why. While trying to do so I noticed Kubernetes workers have their memory accumulated over time, eventually triggering the OOM Killer.

Here are the logs from the Kubernetes worker before getting terminated. They are a pair of "connection closed" and "onnection successfully authorized" for one test session

{"id":"oJx7TKi9mR","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Q997QWACl4","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:46.407128203Z"}
{"id":"449jaHMm5T","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_jqgyLTSziv","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:46.965141635Z"}
{"id":"tErxS2SIQ1","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_jqgyLTSziv","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:47.32770584Z"}
{"id":"495Ix5buYU","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CQ9iY6Myke","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:47.340337875Z"}
{"id":"Pe5PhHYiBa","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CQ9iY6Myke","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:47.718110683Z"}
{"id":"hTiF9eHvG0","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_874OjNeO4y","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:48.000464653Z"}
{"id":"Nbo758Q23Q","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_874OjNeO4y","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:48.364532024Z"}
{"id":"bjcAmFwwYE","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_wVdGwPcccD","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:48.371766627Z"}
{"id":"2XTV1qevv1","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_wVdGwPcccD","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:48.727536163Z"}
{"id":"5MxYnnVZ0C","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_9haSzX6YJJ","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:49.002891713Z"}
{"id":"b3k7bcLQeJ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_9haSzX6YJJ","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:49.345464557Z"}
{"id":"zbxdUAFy7p","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ekDldw13Eo","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:49.396608915Z"}
{"id":"olX5WeYXAx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ekDldw13Eo","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:49.738106996Z"}
{"id":"GBKl2MfVe9","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CADLMSgJhV","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:50.01555931Z"}
{"id":"ajNsejjqps","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_oixNPJOaMu","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:50.481123678Z"}
{"id":"sIIIPwTv5W","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CADLMSgJhV","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:50.494869057Z"}
{"id":"3GfQXTdyon","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_oixNPJOaMu","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:50.892308222Z"}
{"id":"l09iBvO54X","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_L6MzNJlZNq","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:51.228680545Z"}
{"id":"yoeI7Fa59T","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Xh5oDjoEJ1","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:51.585412151Z"}
{"id":"CFqqICnkjx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_L6MzNJlZNq","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:51.607817399Z"}
{"id":"wn58twTZNx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Xh5oDjoEJ1","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:51.925164462Z"}
{"id":"yxwCWJ5Ofn","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_5kTU0yVZ73","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:52.29254077Z"}
{"id":"TrPM48780k","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_JWJKVfUyWc","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:52.677250379Z"}
{"id":"JNT8jrbPJt","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_5kTU0yVZ73","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:52.84057639Z"}
{"id":"Ku632IM3JU","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_JWJKVfUyWc","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:53.046395768Z"}
{"id":"stPkR3AaR5","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_xO9zIaiLVs","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:53.562773662Z"}
{"id":"0OuGg7a2uR","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_00VVxpHkI1","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:53.836104378Z"}
{"id":"sE63aJWPJW","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_xO9zIaiLVs","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:53.919138614Z"}
{"id":"rjBN4igg1L","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_00VVxpHkI1","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:54.237129584Z"}
{"id":"nm0I8XPbAv","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_NBhNPtuUFP","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:54.591907279Z"}
{"id":"9yfK3IxKYD","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mNjYX6KZUh","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:54.910315784Z"}
{"id":"CIlep55xCE","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_NBhNPtuUFP","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:55.011421927Z"}
{"id":"NfD2qbwnVg","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mNjYX6KZUh","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:55.258102817Z"}
{"id":"dQpdO7aL5N","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_v4CNM0LKCc","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:55.72135088Z"}
{"id":"GSCwrJnsse","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ugpj3dFEnt","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:56.014455103Z"}
{"id":"PF0OBygXgi","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_v4CNM0LKCc","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:56.108128962Z"}
{"id":"GbbRPOpXEx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ugpj3dFEnt","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:56.35724017Z"}
{"id":"GGwhLT1mYZ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_t5OjYUCH4z","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:56.795964659Z"}
{"id":"m6S5WksiBn","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CoTs0uLacS","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:57.099168449Z"}
{"id":"o6nRey02t9","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_t5OjYUCH4z","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:57.154168202Z"}
{"id":"lKilZvrjnH","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CoTs0uLacS","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:57.432855529Z"}
{"id":"ZgPX0ewI7D","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_2YKJOkkeLJ","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:57.839222847Z"}
{"id":"OtscSx9FYX","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hP1IUNpuL2","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:58.126327902Z"}
{"id":"cGRiFPMDKm","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_2YKJOkkeLJ","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:58.300395807Z"}
{"id":"84j01Y3FaF","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hP1IUNpuL2","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:58.463641656Z"}
{"id":"rRZlWgpAZM","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_sS0Y7fp2Fn","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:58.978636214Z"}
{"id":"sCwR61VGgI","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_sS0Y7fp2Fn","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:59.32753875Z"}
{"id":"mjZswHvFzD","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_600SwVdBQn","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:46:59.34067157Z"}
{"id":"kl13GxddyQ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_eZjQcquk3n","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:00.174106925Z"}
{"id":"Gfy6eYccrq","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_600SwVdBQn","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:00.318133872Z"}
{"id":"luUlNILTID","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_eZjQcquk3n","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:00.598838913Z"}
{"id":"tXVh9gL8Oc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_BzRnckdm1V","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:01.086056191Z"}
{"id":"gu0vyVNcGC","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6ZQ55pI4Xj","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:01.40226454Z"}
{"id":"ZJsxDnwcRj","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_BzRnckdm1V","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:01.425368737Z"}
{"id":"w7qIYvzzO5","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6ZQ55pI4Xj","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:01.749435007Z"}
{"id":"eZIxW2AWto","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_2rqg7QJ3Gx","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:02.14674244Z"}
{"id":"rLJEvoGbPb","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_EczXWUk68a","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:02.470678149Z"}
{"id":"QblJQYDQpd","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_2rqg7QJ3Gx","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:02.489230583Z"}
{"id":"xU4OuK7N5d","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_EczXWUk68a","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:02.818776247Z"}
{"id":"oY4px2HQYn","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_nuNiNNI09B","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:03.185448707Z"}
{"id":"JMxxKh9NYG","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_0Uz7RXmB5J","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:03.506004737Z"}
{"id":"9T1aDPnwIL","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_nuNiNNI09B","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:03.534520186Z"}
{"id":"gez6WBuAnR","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_0Uz7RXmB5J","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:03.854029764Z"}
{"id":"lJW1BNkcX5","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_85vIgmbq7B","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:04.228987879Z"}
{"id":"w9aEuvsChi","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ErkTZ6LmBL","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:04.563645031Z"}
{"id":"8QDPjvpfVc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_85vIgmbq7B","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:04.587404424Z"}
{"id":"xIXfqAz9DB","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ErkTZ6LmBL","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:05.021328149Z"}
{"id":"5ZtaPeVONV","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_1hbAE2sTNR","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:05.267836296Z"}
{"id":"E989reO2tR","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_1hbAE2sTNR","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:05.623374498Z"}
{"id":"myYjDexxCr","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_NUnDzPELGM","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:05.702093965Z"}
{"id":"UGnHtgotyp","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_NUnDzPELGM","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:06.092111403Z"}
{"id":"00G7b8rZvx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_69XKzutorI","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:06.323980362Z"}
{"id":"o7DQjioh2Y","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_69XKzutorI","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:06.661015762Z"}
{"id":"N6Aehidahw","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_3HIgfZrPu1","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:06.837962459Z"}
{"id":"9fRuoiwKwg","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_3HIgfZrPu1","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:07.238923095Z"}
{"id":"jxXw6hA0fx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mdDJPjIt7L","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:07.458951278Z"}
{"id":"EkhEhWTklc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mdDJPjIt7L","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:07.829784328Z"}
{"id":"zO1ZTRdtGU","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6zWW1eJAZD","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:07.901417242Z"}
{"id":"Fq7dHmdkyJ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6zWW1eJAZD","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:08.299803348Z"}
{"id":"Nqv803B3QU","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_1c5YT91nZI","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:08.534886474Z"}
{"id":"9EPdIncE0u","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_1c5YT91nZI","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:08.895934161Z"}
{"id":"HnkwtpmzWr","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hjgLMaxGLK","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:09.049800822Z"}
{"id":"MPC5bONMTP","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hjgLMaxGLK","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:09.502439343Z"}
{"id":"Fou9Lm49rW","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_RNJhAg8sr3","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:09.577331616Z"}
{"id":"pJOoPmALUY","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_RNJhAg8sr3","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:10.109230699Z"}
{"id":"3pkHMZXjWV","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_cHOlNAamtD","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:10.173292286Z"}
{"id":"XRDKtaGVNQ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_cHOlNAamtD","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:10.786170744Z"}
{"id":"Y2dBHV9orK","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6H2Tqvu7oO","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:10.809772395Z"}
{"id":"pk9UrsO2mc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6H2Tqvu7oO","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:11.192777907Z"}
{"id":"VCTW9aU9re","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_eg2vCPLI0Y","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:11.476766927Z"}
{"id":"lpSHRjyyR0","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_eg2vCPLI0Y","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:11.826691048Z"}
{"id":"kvOpJ9Zq8Z","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_YGhEDeWSzd","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:11.917783368Z"}
{"id":"2UWa7PISNC","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_YGhEDeWSzd","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:12.290586732Z"}
{"id":"KKL2D6jkqo","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ep7lgjb193","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:12.528097338Z"}
{"id":"EJ6DIWz28d","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ep7lgjb193","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:12.878428354Z"}
{"id":"wkk2uWZiCh","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MqTEmjVl6w","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:12.970712717Z"}
{"id":"hgxZkCuObo","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MqTEmjVl6w","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:13.311587844Z"}
{"id":"lGp29tYZ5j","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MhzbumaJmp","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:13.556613446Z"}
{"id":"Api0xca5wH","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MhzbumaJmp","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:13.953257444Z"}
{"id":"X5EgC99SxL","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ZOd7O9R2iU","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:14.04841059Z"}
{"id":"zusZd23oLu","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ZOd7O9R2iU","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:14.404348031Z"}
{"id":"vKHD4lQ7UL","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mbvhPSukEk","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:14.674409532Z"}
{"id":"YaomzKah1b","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_gTz5wfx6Cw","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:15.112971442Z"}
{"id":"hifKPsBSPf","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_mbvhPSukEk","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:15.129819956Z"}
{"id":"mwiTMYjxr4","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_gTz5wfx6Cw","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:15.508863933Z"}
{"id":"LWv7uheirW","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_K0jikNV9AG","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:15.772297546Z"}
{"id":"Q2qx9SZXMa","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Fm6mYSwfe6","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:16.212213683Z"}
{"id":"qOX0QxywM9","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_K0jikNV9AG","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:16.295074517Z"}
{"id":"CRzwWxHire","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Fm6mYSwfe6","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:16.563208176Z"}
{"id":"LxmYAlpzOK","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_YeJ1tiueCq","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:17.104883706Z"}
{"id":"5sB7Go0sKG","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ppaQzwIvAa","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:17.336108739Z"}
{"id":"aT6lXe0nMJ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_YeJ1tiueCq","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:17.478274127Z"}
{"id":"Ln5yiDBTnx","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_ppaQzwIvAa","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:17.745162471Z"}
{"id":"wysL9DL8Ip","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_PbOjyCrtn5","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:18.275964934Z"}
{"id":"lH8EgWkijn","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_b7twutt76o","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:18.576265184Z"}
{"id":"4oiM6aUyEC","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_PbOjyCrtn5","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:18.631785482Z"}
{"id":"Rnd4xgfcAc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_b7twutt76o","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:18.922808045Z"}
{"id":"WQ7gKH4Oc4","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Eb88A9R00E","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:19.320648077Z"}
{"id":"hLBEWsvKlB","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MbvMNlIXkY","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:19.618165184Z"}
{"id":"WNIOAN3Erw","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Eb88A9R00E","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:19.664993733Z"}
{"id":"M0OCI6Ko3m","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_MbvMNlIXkY","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:19.963562552Z"}
{"id":"xiiqgNXY19","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_DO5Cscvg58","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:20.372495433Z"}
{"id":"WxCvs47PqS","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_j2vN6Iv84s","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:20.683373974Z"}
{"id":"IiuCd6O0EC","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_DO5Cscvg58","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:20.809513966Z"}
{"id":"OZzGanoqjE","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_j2vN6Iv84s","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:21.033033102Z"}
{"id":"wfNvcdEUiJ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_WCFx8nillO","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:21.525182761Z"}
{"id":"SSh0ZUvruT","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Zy9Yt4FXVX","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:21.821369043Z"}
{"id":"4GE1CP4LBp","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_WCFx8nillO","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:21.907448144Z"}
{"id":"QfrXVIkVdI","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Zy9Yt4FXVX","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:22.154835094Z"}
{"id":"oFhDpzZII5","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6GdZCpPU69","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:22.654325476Z"}
{"id":"EwKsyySlqX","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hmtvoVm5sG","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:22.890513707Z"}
{"id":"WI9wTP4RAj","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_6GdZCpPU69","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:23.025646194Z"}
{"id":"cUrX3B5s2x","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_hmtvoVm5sG","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:23.236763895Z"}
{"id":"V8lE3sXjtA","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_gWqYOxlU8y","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:23.686831349Z"}
{"id":"LfvI7wNBbT","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_80BHfR9c77","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:23.954364891Z"}
{"id":"YlozikXsQK","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_gWqYOxlU8y","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:24.096992744Z"}
{"id":"9mNhrlBBOg","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_80BHfR9c77","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:24.302635563Z"}
{"id":"S0eosxXegS","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_OfyE0cMp9b","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:24.770264409Z"}
{"id":"QpnSS1TNiZ","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Elb7Q1yXuW","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:25.035956749Z"}
{"id":"rfjYKYlE13","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_Elb7Q1yXuW","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:25.383642964Z"}
{"id":"VhvXyO8Hyh","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_OfyE0cMp9b","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:25.68103456Z"}
{"id":"ZOYHtusX5Q","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CUkewZDMmx","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:26.062664675Z"}
{"id":"Gf99lmPRqY","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_WJes5TiZVy","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:26.371166169Z"}
{"id":"wXk47ph9RM","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_CUkewZDMmx","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:26.403820915Z"}
{"id":"5N5GNeToBc","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_WJes5TiZVy","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:26.736663548Z"}
{"id":"jptbASuTSP","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_pUSkIYuC6S","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:27.106581661Z"}
{"id":"qRQnxN7uwN","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_KTvh2eY08m","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:27.428112038Z"}
{"id":"jb3TW7sNiO","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_pUSkIYuC6S","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:27.457791449Z"}
{"id":"ty6ffPjjod","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_KTvh2eY08m","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:27.784475936Z"}
{"id":"8ABxuRIOiD","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_qCAoNQT9n8","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:28.192810263Z"}
{"id":"8atXt5oQUP","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_dbqiyqBvW3","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:28.493825774Z"}
{"id":"QnC6aL3kRu","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_qCAoNQT9n8","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:28.594476694Z"}
{"id":"x1yrYLBGQw","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_dbqiyqBvW3","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:29.037317269Z"}
{"id":"4Ly0cLdRNj","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_z5ArSpDkzf","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:29.31244687Z"}
{"id":"yPH9ZFSHoK","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_z5ArSpDkzf","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:29.757115898Z"}
{"id":"zbggwq1yzk","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_wSkDK9irN4","msg":"connection successfully authorized","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:29.758644135Z"}
{"id":"vpJTQXBYlp","source":"https://hashicorp.com/boundary/boundary-worker-5c96478d98-l75rg/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_wSkDK9irN4","msg":"connection closed","session_id":"s_LWNWWP2L5l"}},"datacontentype":"application/cloudevents","time":"2024-06-01T11:47:30.129418319Z"}

The logs from Boundary CLI:

Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": EOF
Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": EOF
Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": read tcp 172.26.138.103:45962->1.234.567.89:9202: read: connection reset by peer
Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": read tcp 172.26.138.103:45960->1.234.567.89:9202: read: connection reset by peer
Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": EOF
error reading handshake result: failed to read protobuf message: failed to get reader: failed to read frame header: EOF
Session credentials were not accepted, or session is unauthorized
Session credentials were not accepted, or session is unauthorized
error reading handshake result: failed to read protobuf message: failed to get reader: failed to read frame header: read tcp 172.26.138.103:40810->1.234.567.89:9202: read: connection reset by peer
Error dialing the worker: failed to WebSocket dial: failed to send handshake request: Get "http://k8s-worker-public-domain.com:9202/v1/proxy": read tcp 172.26.138.103:40826->1.234.567.89:9202: read: connection reset by peer

To Reproduce Steps to reproduce the behavior:

  1. Connect to a Boundary target behind a Boundary Kubernetes worker
  2. Notice the initial memory usage for the worker pod
  3. Load-test the worker with a simple loop and keep it running.
  4. After an hour or so, if the worker did not get OOMKilled, terminate the loop
  5. Notice the pod's memory. You should find it relatively high than the initial memory, and it doesn't get lower with time.

In my case, the simple loop got the pod OOMKilled in 2 and half hours:

    State:          Running
      Started:      Sat, 01 Jun 2024 18:01:39 +0300
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Sat, 01 Jun 2024 15:22:30 +0300
      Finished:     Sat, 01 Jun 2024 18:01:38 +0300
    Ready:          True
    Restart Count:  2
    Limits:
      memory:  1Gi
    Requests:
      cpu:     50m
      memory:  256Mi

Expected behavior Worker should free its unused memory after cancelling the session.

Additional context Worker version: v0.15.4 Controller version: v0.14.3 CLI version: 0.14.2

rfc2119 avatar Jun 01 '24 16:06 rfc2119