Support for tools like telefunc

[magne4000]

Describe the bug

Telefunc allows simple frontend to backend calls, avoiding direct call to fetch.

// The `CreateTodo.telefunc.js` file is not loaded:
// Telefunc transforms `*.telefunc.js` imports into a
// thin HTTP client.
import { onNewTodo } from './CreateTodo.telefunc.js'

...

await onNewTodo({ text })

Using telefunc + sveltekit can cause the following issues:

• Cannot import $env/static/private into client-side code
• Cannot import $lib/server/xyz.telefunc.ts into client-side code

Those errors happen at build time and in dev mode, but in dev mode, switching between pages can make it work again. See reproduction repo

Reproduction

https://github.com/magne4000/sveltekit-telefunc-repro

pnpm i && pnpm run dev

The 3 links at the top have the 3 following cases:

• working
• failing because of $env/static/private import
• failing because of $lib/server

Logs

[vite-plugin-svelte-kit] Cannot import $lib/server/test3.telefunc.ts into client-side code:
- .svelte-kit/generated/nodes/4.js
  - src/routes/failing/server-folder/+page.svelte
    - $lib/server/test3.telefunc.ts
error during build:
Error: Cannot import $lib/server/test3.telefunc.ts into client-side code:
- .svelte-kit/generated/nodes/4.js
  - src/routes/failing/server-folder/+page.svelte
    - $lib/server/test3.telefunc.ts
    at IllegalModuleGuard.assert_legal (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:47:11)
    at IllegalModuleGuard.assert_legal (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:49:9)
    at prevent_illegal_rollup_imports (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:424:8)
    at file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/index.js:357:7
    at Array.forEach (<anonymous>)
    at Object.handler (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/index.js:349:25)
    at file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/es/shared/rollup.js:22710:40
    at async PluginDriver.hookParallel (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/es/shared/rollup.js:22632:17)
    at async file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/es/shared/rollup.js:23763:13
    at async catchUnfinishedHookActions (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/es/shared/rollup.js:23088:20)

Cannot import $env/static/private into client-side code:
- src/routes/failing/private-env/+page.svelte
  - $lib/test2.telefunc.ts
    - $env/static/private
Error: Cannot import $env/static/private into client-side code:
- src/routes/failing/private-env/+page.svelte
  - $lib/test2.telefunc.ts
    - $env/static/private
    at IllegalModuleGuard.assert_legal (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:47:11)
    at IllegalModuleGuard.assert_legal (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:49:9)
    at prevent_illegal_vite_imports (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/utils.js:437:8)
    at Object.result.component (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:93:9)
    at async Promise.all (index 1)
    at async render_response (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/runtime/server/page/render.js:82:16)
    at async render_page (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/runtime/server/page/index.js:276:10)
    at async resolve (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/runtime/server/index.js:231:17)
    at async respond (file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/runtime/server/index.js:301:20)
    at async file:///home/magne/workspace/sveltekit-telefunc/node_modules/.pnpm/@[email protected][email protected][email protected]/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:406:22

System Info

System:
    OS: Linux 5.19 undefined
    CPU: (16) x64 AMD Ryzen 7 5800X 8-Core Processor
    Memory: 23.40 GB / 31.26 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 16.15.0 - ~/.volta/tools/image/node/16.15.0/bin/node
    Yarn: 1.22.19 - ~/.volta/tools/image/node/16.15.0/bin/yarn
    npm: 8.10.0 - ~/.volta/tools/image/npm/8.10.0/bin/npm
  Browsers:
    Firefox: 104.0.2
  npmPackages:
    @sveltejs/adapter-auto: next => 1.0.0-next.72
    @sveltejs/kit: next => 1.0.0-next.480
    svelte: ^3.44.0 => 3.50.1
    vite: ^3.1.0 => 3.1.0

Severity

serious, but I can work around it

Additional Information

The .telefunc. files are untouched on server side, but are obviously transformed when used client side. There seems to be a mixup in the dependencies check. I don't know how difficult of a fix this is, but a hacky workaround could be to exclude .telefunc. files from the check :shrug:.

[magne4000]

cc @brillout

[brillout]

Author of Telefunc here. If there is anything I can do to help, let me know.

Note that RPC is on the rise: tRPC has 70k weekly downloads and growing.

Telefunc is similar to tRPC but with a much cleaner design.

I think enabling SvelteKit users to use an RPC solution like Telefunc adds real and significant value.

I'm looking forward to the SvelteKit + Telefunc combo.

[Conduitry]

Is the idea to use this during SSR as well? The would be a pretty big change and I honestly have no idea what that would look like yet, because SvelteKit currently figures out which responses it needs to serialize into the HTML by having all of the API requests go through its wrapped fetch function. If fetch is skipped, it can't figure that out.

Or is the plan to only use this for requests from the browser, and the issue is that SK is looking at untransformed modules (or something) and sees a client-side import of a module that should be server-only?

[magne4000]

Is the idea to use this during SSR as well?

I guess so, but what I'm actually doing is importing the .telefunc. files I need in +page.server.ts and pass the result as props. I think that it would be great if it could work, but it's IMO less important than the second point.

Or is the plan to only use this for requests from the browser, and the issue is that SK is looking at untransformed modules (or something) and sees a client-side import of a module that should be server-only?

That's currently the issue most of people will face, and that's what is reported here :+1:

[elliott-with-the-longest-name-on-github]

This is intended behavior. You can't import anything from $lib/server into client-side code. Same for $env/*/private. The errors are showing you the import chain that's causing the problem.

[brillout]

Or is the plan to only use this for requests from the browser

Yea I think we can focus on this for now and then see later if we want to also support fetching SSR data.

[magne4000]

You can't import anything from $lib/server into client-side code. Same for $env/*/private. The errors are showing you the import chain that's causing the problem.

The thing is that the file itself is not imported into client-side code, it's replaced by something else. Conceptually I could understand that we would still not want to "see" $lib/server imports written in client-side code (whether there is actually server side code in it or not), but for $env/*/private it's a real issue.

I have actual use-cases where I need private env vars in my .telefunc. files (DB connection), they will never appear in client-side code, but I still get an error. For now I had to use process.env. to circumvent the issue.

[brillout]

@tcc-sejohnson Telefunc transforms .telefunc.js imports into a thin client making HTTP requests, see https://telefunc.com/. So while the code of .telefunc.js files is server code, the imports don't actually load the file.

[elliott-with-the-longest-name-on-github]

I don't think there's really any way for us to help with this from an environment variable standpoint. The idea behind telefunc is cool, but the fact is you are importing a server-only module on the client. We don't really have any way of knowing that you're going to screw with it in order to split it.

I think this would have to somehow be resolved from the telefunc side -- you'd have to compile your separate modules prior to the SvelteKit Vite plugin. @Rich-Harris, any ideas?

[brillout]

One solution I see is Telefunc's Vite plugin setting some kind of config, e.g. vite.config.js#svelteKit.allowServerImportFromClient = (filename) => filename.endsWith('.telefunc.js'), for SvelteKit to pick up.

Btw. Telefunc was on Hacker News's front page earlier today. While Telefunc is still young, I think there is interest in RPC solutions like Telefunc and many have asked whether Telefunc supports SveleKit in the past.

I'm very much looking forward to have Telefunc support SvelteKit!

[elliott-with-the-longest-name-on-github]

I'm going to hazard a guess that that solution is going to be a "no" from Rich. I think the "correct" solution here would be to use a public environment variable. You're trying to enable importing a private variable on the client. SvelteKit has no way of knowing that you're "not really" importing it on the client, nor do I really think from a platform perspective that we'd want to trust other libraries when they say "trust me bro, it might look like a client import, but it's not" -- because there's no way of actually verifying it. If you want your environment variables to be importable from client files, they need to be public.

Alternatively, telefunc could provide its own environment variable module in its own Vite plugin. Imagine this:

import { TELEFUNC_API_KEY } from `$telefunc/static` // throws if imported from anywhere but a `.telefunc.` file

If you'd like to take a look at that route, I'd be happy to contribute the code. I could modify a personal Vite plugin I built and steal some code from Kit.

[brillout]

I didn't look at the reproduction, so I may miss something, but I think we are conflating two issues? AFAIK (but I may miss something) the only thing Telefunc needs here is the client to be able to import from the server, since that's, well, the whole idea of Telefunc.

[elliott-with-the-longest-name-on-github]

Incorrect. Telefunc can already be imported from the server on the client. It can't be imported on the client if the server file is accessing private environment variables.

[magne4000]

I think this would have to somehow be resolved from the telefunc side -- you'd have to compile your separate modules prior to the SvelteKit Vite plugin

This kinda breaks the whole point of Telefunc if I understand correctly, as it would require to use different imports on server and client (EDIT: let me see if I can do something with virtual modules 🤔).

SvelteKit has no way of knowing that you're "not really" importing it on the client, nor do I really think from a platform perspective that we'd want to trust other libraries when they say "trust me bro, it might look like a client import, but it's not" -- because there's no way of actually verifying it

I understand the point, but I do not fully agree with it. Yes it seems weird to add some kind of whitelist directly in SvelteKit code, but a specific configuration as mentioned by @brillout does circumvent that. SvelteKit safety checks are good enough without being perfect. This new configuration option would not break SvelteKit in any way, it just allows developers to tell SvelteKit that they know what they are doing.

In any case, I'll keep searching for an alternate solution.

---

I would like to give a real life example of what actually happens on one of my project with SvelteKit+Telefunc.

Telefunc accesses backend stuffs such as:

• DB credentials and query executions
• External API calls with Secrets

Many of those files should reside in $lib/server in a SvelteKit project. In my case, I had to rename src/lib/server to src/lib/_server to be able to compile anything. What this mean is that I had to disable the whole SvelteKit check in order not to micro manage files that were used both by SvelteKit and Telefunc (it's more than 50% of them anyway).

Same thing for $env/static/private, I had to replace the vast majority of those by either process.env or import.meta.env.

The finality is that the added safety of SvelteKit is now almost totally disabled.

[benmccann]

I tried debugging this and kept getting a warning about @brillout/json-s:

@brillout/json-s doesn't appear to be written in CJS, but also doesn't appear to be a valid ES module (i.e. it doesn't have "type": "module" or an .mjs extension for the entry point). Please contact the package author to fix.

I filed https://github.com/brillout/json-serializer/issues/9

[benmccann]

During build, it's only the file in $lib/server that fails. Using $env/static/private is fine. The solution here is pretty simple: don't name your folder server

There might be a bug in dev though

[benmccann]

I believe the SvelteKit code is working as intended here. Telefunc returns the following from transform:

import { PRIVATE_KEY } from '$env/static/private'\nimport { shield as __shieldGenerator_shield } from \"telefunc\";\n\nexport async function test2(subject: string) {\n  return 'test2: ' + PRIVATE_KEY + ' ' + subject;\n}\n\nconst __shieldGenerator_t = __shieldGenerator_shield.type;\n__shieldGenerator_shield(test2, [__shieldGenerator_t.string], { __autoGenerated: true })\n

It's using the PRIVATE_KEY, which is what we're trying to prevent. If it's expected that telefunc does not use the private key on the client, then I think this bug is primarily in telefunc as the removal of the key doesn't appear to be happening.

It actually uses PRIVATE_KEY in build mode as well, so I'm surprised it doesn't fail on that. @tcc-sejohnson I think there might be a bug here in SvelteKit and that we should actually be stricter and are somehow missing a case.

[elliott-with-the-longest-name-on-github]

@benmccann Can you share your efforts at repro-ing?

[brillout]

I believe the SvelteKit code is working as intended here. Telefunc returns the following from transform:

import { PRIVATE_KEY } from '$env/static/private'\nimport { shield as __shieldGenerator_shield } from \"telefunc\";\n\nexport async function test2(subject: string) {\n  return 'test2: ' + PRIVATE_KEY + ' ' + subject;\n}\n\nconst __shieldGenerator_t = __shieldGenerator_shield.type;\n__shieldGenerator_shield(test2, [__shieldGenerator_t.string], { __autoGenerated: true })\n

It's using the PRIVATE_KEY, which is what we're trying to prevent.

That's the server-side transform so it's expected that PRIVATE_KEY is still in there.

The client-side transform does nuke the whole file, including PRIVATE_KEY.

[magne4000]

During build, it's only the file in $lib/server that fails. Using $env/static/private is fine

~~:thinking: didn't saw that at first: it still fails (even for $env/static/private), but only if you reload the page.~~ Nevermind, during the build I indeed have the same behaviour.

[benmccann]

@benmccann Can you share your efforts at repro-ing?

My previous statement may have been incorrect given the correction from @brillout (https://github.com/sveltejs/kit/issues/6700#issuecomment-1301728993). There is still an issue here though which I'll describe below and which you can reproduce simply by using the reproduction link in the issue description.

That's the server-side transform so it's expected that PRIVATE_KEY is still in there.

Ah, right you are. I see transform being called with { ssr: true }, so I guess that's the bug. @tcc-sejohnson the prevent_vite_illegal_imports is getting the graph using a resolve function:

https://github.com/sveltejs/kit/blob/822d283437267a42c063ee729cba4032289b1e18/packages/kit/src/exports/vite/dev/index.js#L90

That resolve function is loading modules for SSR:

https://github.com/sveltejs/kit/blob/822d283437267a42c063ee729cba4032289b1e18/packages/kit/src/exports/vite/dev/index.js#L52

But if this is a client-side check we need to not load them for SSR, but for the client. That's why this is failing for telefunc.

[brillout]

I tried debugging this and kept getting a warning about @brillout/json-s:

@brillout/json-s doesn't appear to be written in CJS, but also doesn't appear to be a valid ES module (i.e. it doesn't have "type": "module" or an .mjs extension for the entry point). Please contact the package author to fix.

I filed brillout/json-serializer#9

👍 Fixed. @magne4000 I made a PR to the reproduction.

[brillout]

During build, it's only the file in $lib/server that fails. Using $env/static/private is fine. The solution here is pretty simple: don't name your folder server

There might be a bug in dev though

I can systematically reproduce this error:

Cannot import $env/static/private into public-facing code:
- src/routes/failing/private-env/+page.svelte 
  - $lib/test2.telefunc.ts 
    - $env/static/private 

Reproduction steps:

git clone [email protected]:magne4000/sveltekit-telefunc-repro && cd sveltekit-telefunc-repro/ && pnpm install && pnpm run dev

Then directly go to http://localhost:5173/failing/private-env. (Don't go to another page before, because otherwise the error won't show.)

This means that The solution here is pretty simple: don't name your folder "server" is, unfortunately, not enough. Telefunc files (i.e. .telefunc.js files) should be able to access private environment variables.