graphql-mesh icon indicating copy to clipboard operation
graphql-mesh copied to clipboard

Cache should interpolate env at runtime

Open Warxcell opened this issue 2 years ago • 4 comments

Issue workflow progress

Progress of the issue based on the Contributor Workflow

Make sure to fork this template and run yarn generate in the terminal.

Please make sure Mesh package versions under package.json matches yours.

  • [ ] 2. A failing test has been provided
  • [ ] 3. A local solution has been provided
  • [ ] 4. A pull request is pending review

Describe the bug

env at cache is interpolated at build time, which makes impossible to run same docker image in different environments.

To Reproduce Steps to reproduce the behavior:

cache:
  redis:
    host: '{env.REDIS_HOST}'
    port: '{env.REDIS_PORT}'

and run mesh build

Expected behavior

{env...} to become process.env.... on build.

Environment:

  • OS: all
  • @graphql-mesh/...: all
  • NodeJS: all

Additional context

image

Warxcell avatar Jun 28 '22 22:06 Warxcell

The image you shared is created at build time and I see it is not interpolated during build time so Mesh will interpolate it during actual runtime.

ardatan avatar Jun 29 '22 06:06 ardatan

Hm, maybe I did something wrong then, yesterday I was very tired. Will try again later today.

Warxcell avatar Jun 29 '22 06:06 Warxcell

Something weird is happening. When using

cache:
  redis:
    host: '{env.REDIS_HOST}'
    port: '{env.REDIS_PORT}'

running it locally (in and out container) - it works. When its deployed to production Kubernetes - it doesn't work anymore.

│ [ioredis] Unhandled error event: Error: connect ETIMEDOUT                                                                                                                                                                                                                                                                         

After changing to

cache:
  redis:
    url: 'redis://{env.REDIS_HOST}:{env.REDIS_PORT}'

Its still working locally (no container) - but image cannot be build.

#17 [builder 8/8] RUN yarn build && yarn tsc
#17 sha256:767d11b51edd14822fce5f20bc170185e6fdfcbc7ed013fa4510cc487a0a0f29
#17 0.302 yarn run v1.22.19
#17 0.313 warning package.json: License should be a valid SPDX license expression
#17 0.322 $ mesh build
#17 1.225 💡 🕸️  Mesh Cleaning existing artifacts
#17 1.233 💡 🕸️  Mesh Reading the configuration
#17 1.500 💥 🕸️  Mesh TypeError [ERR_INVALID_URL]: Invalid URL
#17 1.500     at new NodeError (node:internal/errors:388:5)
#17 1.500     at UR...

Warxcell avatar Jun 29 '22 20:06 Warxcell

Make sure you have environment variables set in your Kubernetes setup.

ardatan avatar Jul 20 '22 17:07 ardatan

I am also facing the same problem (even thought in a slightly different flavour than @Warxcell) In short:

  1. I provide my .meshrc config as:
cache:
  redis:
    host: "${REDIS_HOST}"
    port: "${REDIS_PORT}"
    password: "${REDIS_PWD}"
  1. I then generate my mesh through graphql-mesh build

  2. When I inspect the code generated (inside .mesh/index.ts), I unexpectedly find plain and simple all my values (below an example of what gets generated)

const cache = new (MeshCache as any)({
    ...({ host: 'redistore.tardis.svc.cluster.local', port: '6379', password: 'defaultRedisPassword' } as any)

It does seem like env variables are not computed at runtime. Is there a way we can push even further our env variables into the generated mesh as actual env variables?

Ideally my team will keep the mesh under source control and... having secrets over there is not really a best practice.

ironest avatar Jan 30 '23 05:01 ironest

You should use {env.REDIS_HOST} instead because ${} is handled by YAML processor while {} is handled by Mesh itself. https://the-guild.dev/graphql/mesh/docs/guides/dynamic-vars

Closing this issue for now. Feel free to create a new one with a reproduction if you have any further issues. Thanks!

ardatan avatar Jan 30 '23 09:01 ardatan

Thank you @ardatan it works now.

ironest avatar Jan 31 '23 01:01 ironest

Hello, I'm facing maybe the same problem when deploying the GraphQL Mesh gateway to a cloud provider through Docker and Kubernetes. In our .meshrc.ts, we have an env variable like this: const plugins: YamlConfig.Config['plugins'] = [ { operationFieldPermissions: { permissions: [{ if: 'context.headers.auth_key == env.AUTH_KEY', allow: ['*'] }] } }, ];

For other env variables in the app, we do use {env.VARIABLE} syntax, for example: endpoint: '{env.API_URL}'

We have an .env file inside Docker image that contain those variables. Locally, I run the docker image locally and it works. However, when deploying with Kubernetes, it seems like those env variables are not computed at run time.

quangchuc avatar Mar 07 '23 21:03 quangchuc