workerd icon indicating copy to clipboard operation
workerd copied to clipboard

Docker error: CODE_MOVED for unknown code block?

Open adronitis opened this issue 2 years ago • 12 comments

Running wrangler dev --port=8000 inside Docker with:

/* eslint-disable */
import { Router } from 'itty-router'

const router = Router()
const routes =  JSON.parse(ROUTES)
const proxyTo = hostname => async request => {

  const url = new URL(request.url);
  const forwardedHost = url.hostname;
  url.hostname = hostname;

  const req = new Request(url, request);
  req.headers.append('X-Forwarded-Host', forwardedHost);

  return fetch(req);
}

routes.forEach(route => {
  router.all(route, proxyTo(FRONTEND_HOST));
});

router.all('/api/*', proxyTo(BACKEND_HOST))
router.all('/*', proxyTo(BACKEND_HOST))

addEventListener('fetch', (e) => {
  e.respondWith(router.handle(e.request))
})

Everything works fine if I run this outside of Docker, but if I put everything in Docker, when I want to do the proxy, I get:

worker-router-worker-1  | ✘ [ERROR] workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |
worker-router-worker-1  |   workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |   workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |   workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |

worker-router-worker-1  | ✘ [ERROR] workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |
worker-router-worker-1  |   workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block?
worker-router-worker-1  |

worker-router-worker-1  | ✘ [ERROR] workerd/server/server.c++:2838: error: Uncaught exception: kj/compat/http.c++:1725: failed: expected headerBuffer.size() < MAX_BUFFER [131072 < 131072]; request headers too large
worker-router-worker-1  |
worker-router-worker-1  |   stack: /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@265008c /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2652620 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2656798 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2622b74
worker-router-worker-1  |
[mf:err] Error: Network connection lost.
[mf:inf] GET /terms 500 Internal Server Error (88ms)

Individual URL for the host (eg. front-end) works fine from Docker. Not exactly sure what the headerBuffer.size() and where that could be? Could this due to some host settings within Docker?

adronitis avatar Dec 05 '23 08:12 adronitis

I'm also getting this error, but not just in Docker. It seems to have been introduced in wrangler 3.18 or 3.19, as 3.17 does not have this issue for me.

johnpyp avatar Dec 11 '23 07:12 johnpyp

I'm also getting this error, but not just in Docker. It seems to have been introduced in wrangler 3.18 or 3.19, as 3.17 does not have this issue for me.

That's strange, our Docker env was using 3.17 but still got this error.

adronitis avatar Dec 11 '23 08:12 adronitis

This message is not really an error, at least not one that's relevant to most people. This is another example where our logging is highlighting bugs meant for the workers runtime team to look into. In this case V8's API for mapping JITed memory in order to construct stack traces isn't working the way we expect. This has been firing occasionally for a long time. There's actually no impact for workerd users since this information is only used when generating crash dumps in our internal codebase.

kentonv avatar Dec 11 '23 15:12 kentonv

Hmm, for me it's the only noticeable difference between my worker responding on 3.17 and hanging on 3.19. Maybe it just coincides with another breaking change rather than the cause?

johnpyp avatar Dec 11 '23 17:12 johnpyp

👋 If it helps in debugging this, I'm reliably seeing this for one set of our tests w/ Miniflare: https://github.com/cdnjs/api-server/actions/runs/7440877000/job/20242359379

MattIPv4 avatar Jan 07 '24 21:01 MattIPv4

TBH we should remove the logging for this error as it's unlikely anyone is going to spend time debugging it... it just doesn't matter that much.

kentonv avatar Jan 08 '24 23:01 kentonv

I got the same workerd/jsg/setup.c++:481: error: CODE_MOVED for unknown code block? error when running wrangler pages dev ./build on 3.22 and 3.17.

Pages site is not served, got thousands of 500 internal server error lines.

tddschn avatar Jan 13 '24 04:01 tddschn

@tddschn Sorry, the CODE_MOVED message has nothing to do with any other problems you're seeing.

kentonv avatar Jan 13 '24 18:01 kentonv

I am still seeing this issue in 3.36.0. Perhaps the priority of this issue can be bumped up a bit? Even if it is benign, it makes spotting other errors/warnings in logs more difficult.

Actually, this warning disappears for me when I remove the --reference-types argument from wasm-bindgen-cli

allsey87 avatar Mar 21 '24 14:03 allsey87

I am starting to see them more often in CI and they seem to coincide with some process crashing. Since whenever I see these, my tests timeout. I know most comments indicate, that they should not be relevant, but could we get some more background info?

KeKs0r avatar Mar 22 '24 00:03 KeKs0r

I am starting to see them more often in CI and they seem to coincide with some process crashing. Since whenever I see these, my tests timeout. I know most comments indicate, that they should not be relevant, but could we get some more background info?

Same here, we concluded the same results. But yeah more background info would indeed be nice.

adronitis avatar Mar 22 '24 07:03 adronitis

got the same error with wrangler 3.39.0

https://gist.github.com/Zxilly/9273aa5909f539db3f0dc8736aaab9bd

> [email protected] dev
> wrangler dev

 ⛅️ wrangler 3.39.0
-------------------
Your worker has access to the following bindings:
- R2 Buckets:
  - IMG_BUCKET: img-bucket
⎔ Starting local server...
[wrangler:inf] Ready on http://127.0.0.1:8787
X [ERROR] workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?

  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?


[wrangler:inf] PUT /test.png 201 Created (555ms)
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [b] open a browser, [d] open Devtools, [l] turn off local mode, [c] clear console, [x] to exit                                                                                                                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Zxilly avatar Mar 28 '24 12:03 Zxilly

trying to batch insert rows into a d1 table when all of a sudden these error messages popped up (on random occassions within the process. the requests are still being completed though but still its irritating since you're debugging until you finally find these statements in this issue, that these are not related to your code. weird.

✘ [ERROR] workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?

  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?
  workerd/jsg/setup.c++:508: error: CODE_MOVED for unknown code block?

can we somehow catch them? or prepend them otherwise from being shown to the log?

gh0sTedBuddy avatar Apr 16 '24 21:04 gh0sTedBuddy

I made a PR (#2037) to remove this log message.

kentonv avatar Apr 19 '24 13:04 kentonv