docker icon indicating copy to clipboard operation
docker copied to clipboard
verified publisher icon calcom

feat(frontend): use runtime NEXT_PUBLIC_WEBAPP_URL to replace build-time frontend value

Open conneryn opened this issue 3 years ago • 5 comments

Here is an attempt at fixing #141.

NEXT_PUBLIC_WEBAPP_URL variable is no longer expected during docker build time. Instead, a "placeholder" value is used, and replaced at runtime.

Let me know if you think this is appropriate, or if you'd like to see any changes.

conneryn avatar Sep 09 '22 00:09 conneryn

I have this on the queue for this morning, thanks!

krumware avatar Sep 16 '22 12:09 krumware

The string replacement doesn't appear to be working on my box. Trying to debug.

In container:

echo "${NEXT_PUBLIC_WEBAPP_URL}" http://localhost:3000

./scripts/replace-placeholders.sh

grep -rnwl 'apps/web/.next/' -e 'NEXT_PUBLIC_WEBAPP_URL_PLACEHOLDER'

apps/web/.next/server/middleware.js
apps/web/.next/server/chunks/2622.js
apps/web/.next/server/chunks/7361.js
apps/web/.next/server/chunks/580.js
apps/web/.next/server/chunks/5715.js
apps/web/.next/server/chunks/2470.js
apps/web/.next/server/chunks/3530.js
apps/web/.next/server/chunks/3298.js
apps/web/.next/server/chunks/3325.js
apps/web/.next/server/chunks/7133.js
apps/web/.next/server/chunks/1022.js
apps/web/.next/server/chunks/9014.js
apps/web/.next/server/chunks/8125.js
apps/web/.next/server/chunks/8131.js
apps/web/.next/server/chunks/949.js
apps/web/.next/server/chunks/6039.js
apps/web/.next/server/chunks/7059.js
apps/web/.next/server/pages/iw/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/iw/v2/apps/categories.html
apps/web/.next/server/pages/iw/auth/error.html
apps/web/.next/server/pages/iw/500.html
apps/web/.next/server/pages/iw/cancel/success.html
apps/web/.next/server/pages/iw/video/no-meeting-found.html
apps/web/.next/server/pages/iw/apps/categories.html
apps/web/.next/server/pages/iw/404.html
apps/web/.next/server/pages/iw/apps.html
apps/web/.next/server/pages/ar/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/ar/v2/apps/categories.html
apps/web/.next/server/pages/ar/auth/error.html
apps/web/.next/server/pages/ar/500.html
apps/web/.next/server/pages/ar/cancel/success.html
apps/web/.next/server/pages/ar/video/no-meeting-found.html
apps/web/.next/server/pages/ar/apps/categories.html
apps/web/.next/server/pages/ar/404.html
apps/web/.next/server/pages/ar/apps.html
apps/web/.next/server/pages/it/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/it/v2/apps/categories.html
apps/web/.next/server/pages/it/auth/error.html
apps/web/.next/server/pages/it/500.html
apps/web/.next/server/pages/it/cancel/success.html
apps/web/.next/server/pages/it/video/no-meeting-found.html
apps/web/.next/server/pages/it/apps/categories.html
apps/web/.next/server/pages/it/404.html
apps/web/.next/server/pages/it/apps.html
apps/web/.next/server/pages/vi/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/vi/v2/apps/categories.html
apps/web/.next/server/pages/vi/auth/error.html
apps/web/.next/server/pages/vi/500.html
apps/web/.next/server/pages/vi/cancel/success.html
apps/web/.next/server/pages/vi/video/no-meeting-found.html
apps/web/.next/server/pages/vi/apps/categories.html
apps/web/.next/server/pages/vi/404.html
apps/web/.next/server/pages/vi/apps.html
apps/web/.next/server/pages/getting-started/[[...step]].js
apps/web/.next/server/pages/sv/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/sv/v2/apps/categories.html
apps/web/.next/server/pages/sv/auth/error.html
apps/web/.next/server/pages/sv/500.html
apps/web/.next/server/pages/sv/cancel/success.html
apps/web/.next/server/pages/sv/video/no-meeting-found.html
apps/web/.next/server/pages/sv/apps/categories.html
apps/web/.next/server/pages/sv/404.html
apps/web/.next/server/pages/sv/apps.html
apps/web/.next/server/pages/v2/auth/login.js
apps/web/.next/server/pages/v2/success.js
apps/web/.next/server/pages/auth/sso/[provider].js
apps/web/.next/server/pages/auth/login.js
apps/web/.next/server/pages/auth/error.js
apps/web/.next/server/pages/auth/logout.js
apps/web/.next/server/pages/auth/signup.js
apps/web/.next/server/pages/auth/setup.js
apps/web/.next/server/pages/ru/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/ru/v2/apps/categories.html
apps/web/.next/server/pages/ru/auth/error.html
apps/web/.next/server/pages/ru/500.html
apps/web/.next/server/pages/ru/cancel/success.html
apps/web/.next/server/pages/ru/video/no-meeting-found.html
apps/web/.next/server/pages/ru/apps/categories.html
apps/web/.next/server/pages/ru/404.html
apps/web/.next/server/pages/ru/apps.html
apps/web/.next/server/pages/sr/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/sr/v2/apps/categories.html
apps/web/.next/server/pages/sr/auth/error.html
apps/web/.next/server/pages/sr/500.html
apps/web/.next/server/pages/sr/cancel/success.html
apps/web/.next/server/pages/sr/video/no-meeting-found.html
apps/web/.next/server/pages/sr/apps/categories.html
apps/web/.next/server/pages/sr/404.html
apps/web/.next/server/pages/sr/apps.html
apps/web/.next/server/pages/pl/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/pl/v2/apps/categories.html
apps/web/.next/server/pages/pl/auth/error.html
apps/web/.next/server/pages/pl/500.html
apps/web/.next/server/pages/pl/cancel/success.html
apps/web/.next/server/pages/pl/video/no-meeting-found.html
apps/web/.next/server/pages/pl/apps/categories.html
apps/web/.next/server/pages/pl/404.html
apps/web/.next/server/pages/pl/apps.html
apps/web/.next/server/pages/es-419/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/es-419/v2/apps/categories.html
apps/web/.next/server/pages/es-419/auth/error.html
apps/web/.next/server/pages/es-419/500.html
apps/web/.next/server/pages/es-419/cancel/success.html
apps/web/.next/server/pages/es-419/video/no-meeting-found.html
apps/web/.next/server/pages/es-419/apps/categories.html
apps/web/.next/server/pages/es-419/404.html
apps/web/.next/server/pages/es-419/apps.html
apps/web/.next/server/pages/pt-BR/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/pt-BR/v2/apps/categories.html
apps/web/.next/server/pages/pt-BR/auth/error.html
apps/web/.next/server/pages/pt-BR/500.html
apps/web/.next/server/pages/pt-BR/cancel/success.html
apps/web/.next/server/pages/pt-BR/video/no-meeting-found.html
apps/web/.next/server/pages/pt-BR/apps/categories.html
apps/web/.next/server/pages/pt-BR/404.html
apps/web/.next/server/pages/pt-BR/apps.html
apps/web/.next/server/pages/ja/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/ja/v2/apps/categories.html
apps/web/.next/server/pages/ja/auth/error.html
apps/web/.next/server/pages/ja/500.html
apps/web/.next/server/pages/ja/cancel/success.html
apps/web/.next/server/pages/ja/video/no-meeting-found.html
apps/web/.next/server/pages/ja/apps/categories.html
apps/web/.next/server/pages/ja/404.html
apps/web/.next/server/pages/ja/apps.html
apps/web/.next/server/pages/pt/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/pt/v2/apps/categories.html
apps/web/.next/server/pages/pt/auth/error.html
apps/web/.next/server/pages/pt/500.html
apps/web/.next/server/pages/pt/cancel/success.html
apps/web/.next/server/pages/pt/video/no-meeting-found.html
apps/web/.next/server/pages/pt/apps/categories.html
apps/web/.next/server/pages/pt/404.html
apps/web/.next/server/pages/pt/apps.html
apps/web/.next/server/pages/nl/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/nl/v2/apps/categories.html
apps/web/.next/server/pages/nl/auth/error.html
apps/web/.next/server/pages/nl/500.html
apps/web/.next/server/pages/nl/cancel/success.html
apps/web/.next/server/pages/nl/video/no-meeting-found.html
apps/web/.next/server/pages/nl/apps/categories.html
apps/web/.next/server/pages/nl/404.html
apps/web/.next/server/pages/nl/apps.html
apps/web/.next/server/pages/cs/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/cs/v2/apps/categories.html
apps/web/.next/server/pages/cs/auth/error.html
apps/web/.next/server/pages/cs/500.html
apps/web/.next/server/pages/cs/cancel/success.html
apps/web/.next/server/pages/cs/video/no-meeting-found.html
apps/web/.next/server/pages/cs/apps/categories.html
apps/web/.next/server/pages/cs/404.html
apps/web/.next/server/pages/cs/apps.html
apps/web/.next/server/pages/ro/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/ro/v2/apps/categories.html
apps/web/.next/server/pages/ro/auth/error.html
apps/web/.next/server/pages/ro/500.html
apps/web/.next/server/pages/ro/cancel/success.html
apps/web/.next/server/pages/ro/video/no-meeting-found.html
apps/web/.next/server/pages/ro/apps/categories.html
apps/web/.next/server/pages/ro/404.html
apps/web/.next/server/pages/ro/apps.html
apps/web/.next/server/pages/404.js
apps/web/.next/server/pages/cancel/[uid].js
apps/web/.next/server/pages/ko/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/ko/v2/apps/categories.html
apps/web/.next/server/pages/ko/auth/error.html
apps/web/.next/server/pages/ko/500.html
apps/web/.next/server/pages/ko/cancel/success.html
apps/web/.next/server/pages/ko/video/no-meeting-found.html
apps/web/.next/server/pages/ko/apps/categories.html
apps/web/.next/server/pages/ko/404.html
apps/web/.next/server/pages/ko/apps.html
apps/web/.next/server/pages/en/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/en/v2/apps/huddle01.html
apps/web/.next/server/pages/en/v2/apps/typeform.html
apps/web/.next/server/pages/en/v2/apps/google-calendar.html
apps/web/.next/server/pages/en/v2/apps/ping.html
apps/web/.next/server/pages/en/v2/apps/wipe-my-cal.html
apps/web/.next/server/pages/en/v2/apps/daily-video.html
apps/web/.next/server/pages/en/v2/apps/exchange.html
apps/web/.next/server/pages/en/v2/apps/apple-calendar.html
apps/web/.next/server/pages/en/v2/apps/closecom.html
apps/web/.next/server/pages/en/v2/apps/campfire.html
apps/web/.next/server/pages/en/v2/apps/caldav-calendar.html
apps/web/.next/server/pages/en/v2/apps/categories.html
apps/web/.next/server/pages/en/v2/apps/routing-forms.html
apps/web/.next/server/pages/en/v2/apps/riverside.html
apps/web/.next/server/pages/en/v2/apps/jitsi.html
apps/web/.next/server/pages/en/v2/apps/raycast.html
apps/web/.next/server/pages/en/v2/apps/around.html
apps/web/.next/server/pages/en/v2/apps/google-meet.html
apps/web/.next/server/pages/en/v2/apps/rainbow.html
apps/web/.next/server/pages/en/v2/apps/categories/automation.html
apps/web/.next/server/pages/en/v2/apps/categories/video.html
apps/web/.next/server/pages/en/v2/apps/categories/web3.html
apps/web/.next/server/pages/en/v2/apps/categories/other.html
apps/web/.next/server/pages/en/v2/apps/categories/calendar.html
apps/web/.next/server/pages/en/v2/apps/whereby.html
apps/web/.next/server/pages/en/v2/apps/n8n.html
apps/web/.next/server/pages/en/auth/error.html
apps/web/.next/server/pages/en/500.html
apps/web/.next/server/pages/en/cancel/success.html
apps/web/.next/server/pages/en/video/no-meeting-found.html
apps/web/.next/server/pages/en/apps/huddle01.html
apps/web/.next/server/pages/en/apps/typeform.html
apps/web/.next/server/pages/en/apps/google-calendar.html
apps/web/.next/server/pages/en/apps/ping.html
apps/web/.next/server/pages/en/apps/wipe-my-cal.html
apps/web/.next/server/pages/en/apps/daily-video.html
apps/web/.next/server/pages/en/apps/exchange.html
apps/web/.next/server/pages/en/apps/apple-calendar.html
apps/web/.next/server/pages/en/apps/closecom.html
apps/web/.next/server/pages/en/apps/campfire.html
apps/web/.next/server/pages/en/apps/caldav-calendar.html
apps/web/.next/server/pages/en/apps/categories.html
apps/web/.next/server/pages/en/apps/routing-forms.html
apps/web/.next/server/pages/en/apps/riverside.html
apps/web/.next/server/pages/en/apps/jitsi.html
apps/web/.next/server/pages/en/apps/raycast.html
apps/web/.next/server/pages/en/apps/around.html
apps/web/.next/server/pages/en/apps/google-meet.html
apps/web/.next/server/pages/en/apps/rainbow.html
apps/web/.next/server/pages/en/apps/categories/automation.html
apps/web/.next/server/pages/en/apps/categories/video.html
apps/web/.next/server/pages/en/apps/categories/web3.html
apps/web/.next/server/pages/en/apps/categories/other.html
apps/web/.next/server/pages/en/apps/categories/calendar.html
apps/web/.next/server/pages/en/apps/whereby.html
apps/web/.next/server/pages/en/apps/n8n.html
apps/web/.next/server/pages/en/404.html
apps/web/.next/server/pages/en/apps.html
apps/web/.next/server/pages/fr/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/fr/v2/apps/categories.html
apps/web/.next/server/pages/fr/auth/error.html
apps/web/.next/server/pages/fr/500.html
apps/web/.next/server/pages/fr/cancel/success.html
apps/web/.next/server/pages/fr/video/no-meeting-found.html
apps/web/.next/server/pages/fr/apps/categories.html
apps/web/.next/server/pages/fr/404.html
apps/web/.next/server/pages/fr/apps.html
apps/web/.next/server/pages/zh-CN/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/zh-CN/v2/apps/categories.html
apps/web/.next/server/pages/zh-CN/auth/error.html
apps/web/.next/server/pages/zh-CN/500.html
apps/web/.next/server/pages/zh-CN/cancel/success.html
apps/web/.next/server/pages/zh-CN/video/no-meeting-found.html
apps/web/.next/server/pages/zh-CN/apps/categories.html
apps/web/.next/server/pages/zh-CN/404.html
apps/web/.next/server/pages/zh-CN/apps.html
apps/web/.next/server/pages/settings/admin.js
apps/web/.next/server/pages/settings/profile.js
apps/web/.next/server/pages/old-getting-started.js
apps/web/.next/server/pages/de/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/de/v2/apps/categories.html
apps/web/.next/server/pages/de/auth/error.html
apps/web/.next/server/pages/de/500.html
apps/web/.next/server/pages/de/cancel/success.html
apps/web/.next/server/pages/de/video/no-meeting-found.html
apps/web/.next/server/pages/de/apps/categories.html
apps/web/.next/server/pages/de/404.html
apps/web/.next/server/pages/de/apps.html
apps/web/.next/server/pages/success.js
apps/web/.next/server/pages/team/[slug]/[type].js
apps/web/.next/server/pages/d/[link]/[slug].js
apps/web/.next/server/pages/es/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/es/v2/apps/categories.html
apps/web/.next/server/pages/es/auth/error.html
apps/web/.next/server/pages/es/500.html
apps/web/.next/server/pages/es/cancel/success.html
apps/web/.next/server/pages/es/video/no-meeting-found.html
apps/web/.next/server/pages/es/apps/categories.html
apps/web/.next/server/pages/es/404.html
apps/web/.next/server/pages/es/apps.html
apps/web/.next/server/pages/_app.js
apps/web/.next/server/pages/api/teams/[team]/upgrade.js
apps/web/.next/server/pages/api/auth/forgot-password.js
apps/web/.next/server/pages/api/trpc/[trpc].js
apps/web/.next/server/pages/zh-TW/v2/settings/teams/new/[[...step]].html
apps/web/.next/server/pages/zh-TW/v2/apps/categories.html
apps/web/.next/server/pages/zh-TW/auth/error.html
apps/web/.next/server/pages/zh-TW/500.html
apps/web/.next/server/pages/zh-TW/cancel/success.html
apps/web/.next/server/pages/zh-TW/video/no-meeting-found.html
apps/web/.next/server/pages/zh-TW/apps/categories.html
apps/web/.next/server/pages/zh-TW/404.html
apps/web/.next/server/pages/zh-TW/apps.html
apps/web/.next/cache/webpack/edge-server-production/0.pack
apps/web/.next/cache/webpack/server-production/0.pack
apps/web/.next/cache/webpack/client-production/0.pack
apps/web/.next/static/chunks/2783-6c4c195bda1cc2d9.js
apps/web/.next/static/chunks/2470.da0e00e7eceacbd3.js
apps/web/.next/static/chunks/6470-f642f830b5d52777.js
apps/web/.next/static/chunks/7339-b5d0a67e69118a8c.js
apps/web/.next/static/chunks/3805-4ad6183e60ca53b7.js
apps/web/.next/static/chunks/7261-ab66d164a5e7b56d.js
apps/web/.next/static/chunks/8032-3d862818e2ecc822.js
apps/web/.next/static/chunks/1781-bd697c0fa3cf242a.js
apps/web/.next/static/chunks/pages/v2/event-types-7d01ddf24c1d2657.js
apps/web/.next/static/chunks/pages/v2/settings/teams/[id]/profile-e836ecc24bd18219.js
apps/web/.next/static/chunks/pages/v2/settings/admin/impersonation-f72247f29ef0ff5f.js
apps/web/.next/static/chunks/pages/auth/setup-7d2cd81b4395e3d3.js
apps/web/.next/static/chunks/pages/auth/login-4f8b8aec784b5834.js
apps/web/.next/static/chunks/pages/auth/signup-fe104769b0a1527c.js
apps/web/.next/static/chunks/pages/auth/logout-6864908aa38cdfb0.js
apps/web/.next/static/chunks/pages/auth/new-e215786ae9a8a14a.js
apps/web/.next/static/chunks/pages/auth/error-09743d07510e1916.js
apps/web/.next/static/chunks/pages/404-e652f94127b271ac.js
apps/web/.next/static/chunks/pages/cancel/[uid]-9f1d02579872a6a8.js
apps/web/.next/static/chunks/pages/cancel/success-cae82dbfb29dd0c2.js
apps/web/.next/static/chunks/pages/video/no-meeting-found-3f174f75d3c88550.js
apps/web/.next/static/chunks/pages/video/meeting-not-started/[uid]-fa72b9def19444ca.js
apps/web/.next/static/chunks/pages/video/meeting-ended/[uid]-89eb336633996612.js
apps/web/.next/static/chunks/pages/settings/teams/[id]-2f7b078afdc29fb9.js
apps/web/.next/static/chunks/pages/settings/admin-9fcec0817da42b24.js
apps/web/.next/static/chunks/pages/team/[slug]-64a22d37d6655191.js
apps/web/.next/static/chunks/pages/_app-1ce36dd30cfe6abe.js
apps/web/.next/static/chunks/8814-9f0685518d90c22d.js
apps/web/.next/static/chunks/1440-e6f43d463253e527.js

krumware avatar Sep 24 '22 15:09 krumware

ok pushed past that, checking a few other mods

krumware avatar Sep 24 '22 16:09 krumware

@conneryn I'd propose changing the placeholder back to the default of http://localhost:3000 that would make it a little bit cleaner for local dev, as well as ease some immutability workflow. That way, if I add the following steps after COPY scripts scripts, we can enable immutable builds that would also be optionally swappable

ARG NEXT_PUBLIC_WEBAPP_URL
ARG CALCOM_TELEMETRY_DISABLE
ENV NODE_ENV production \
    NEXT_PUBLIC_WEBAPP_URL=${NEXT_PUBLIC_WEBAPP_URL:-http://localhost:3000}
RUN scripts/replace-placeholders.sh

(I have Skaffold configs coming shortly, which this would help support)

If possible, I'd also like to see some input parameters on that start.sh script to make the call to replace-placeholders an opt-in, to be passed in via the docker-compose file and a runtime command: override

krumware avatar Sep 24 '22 17:09 krumware

Thanks for the feedback! How about something like this?

  1. I added back the ARG for NEXT_PUBLIC_WEBAPP_URL, so containers can be created with a specific value set at build-time.
  2. We use a RUN of scripts/replace-placeholder.sh as the last step of the Dockerfile build to actually set the values. This way, rebuilding the container with different NEXT_PUBLIC_WEBAPP_URL will be very fast (all other layers stay unchanged).
  3. We store the NEXT_PUBLIC_WEBAPP_URL used during the build stage under an environment variable (BUILT_NEXT_PUBLIC_WEBAPP_URL). At run-time, we can compare the run-time specified NEXT_PUBLIC_WEBAPP_URL with the value used at build time, and ONLY do the find/replace if they differ.

I did not add an explicit opt-in/out of the replace-placeholders, but I feel it's not really necessary anymore.

conneryn avatar Sep 29 '22 20:09 conneryn

Several notes, non-blocking, but want to include here:

Having an issue with the NEXTAUTH_URL when changing to anything other than localhost, when testing locally. this is a known internal issue,

In previous testing, I also needed to add directories to the find replace find apps/web/.next/ apps/web/public packages/embeds node_modules/.cache/turbo -type f | (after retesting, not sure if still needed, also noting)

Some container and kubernetes security tools may block the use of find at runtime, so either an opt-in or a skip flag may still be needed

krumware avatar Nov 08 '22 13:11 krumware

retesting now after related fix: https://github.com/calcom/cal.com/pull/5419

krumware avatar Nov 08 '22 16:11 krumware

will add instructions related to NEXTAUTH fixes in another PR

krumware avatar Nov 08 '22 18:11 krumware

I'm not sure if I should open a new PR, but this doesn't work for my use case with a reverse proxy. I'm using nginx reverse proxy to handle the url and letsencrypt and then just want to host this at localhost:3000 but then I want all the links to go to myurl.domain.com/link rather than localhost:3000/link. Is there a way to separate the hosting domain and the url link address?

codyrigg avatar Feb 15 '23 20:02 codyrigg

What you're describing that you want, is how it currently works from a networking standpoint. The issue is that the URLs are embedded in in the static site files, so you need to rewrite those URLs for the static files. There is no hostname matching happening at the cal.com level otherwise, so traffic will not be blocked if you host at localhost:3000 but the static files were rewritten to include your myurl.domain.com/link. The challenge would be if you wanted to use the same workload to serve two different public domains, the static files would only reflect whichever URL you provided to the env vars of the workload at runtime.

For reference, I'm using nginx as a reverse proxy with letsencrypt for certificates, and providing the webapp_url which is replaced at runtime.

krumware avatar Feb 15 '23 20:02 krumware