convex-backend
convex-backend copied to clipboard
π‘Feature Proposal: Allow escaped files and folders
Hi,
First of all, huge thanks to anyone involved in this project. This is a real gem and a pleasure to use.
The feature I'd like to see added to Convex is a way to escape folders inside the convex folder. In order to have more control on their organization and the api generated.
Situation
Let's say I have a user table and I'd like its api to be:
api.user.login
api.user.checkout.{create,resume}
api.user.crud.{get,create,update,drop}
The only way to get such a result would be with this structure:
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
ββ user.ts <!-- {login} -->
This has three drawbacks (or more that I didn't think about):
user.tslives out of theuserfolder.- Both names have to be kept in sync to correctly resolve the
api. - All
api.user.*endpoints have to live inuser.ts. If I declareapi.user.loginandapi.user.notRelatedToLogin, both would have to be in the same file. I can't separate them by context.
It quickly becomes a mess as the number of endpoints increases.
Proposal
Allow for a naming syntax that will escape the file or folder's name upon resolving the api. I use () to mimic Next.js routing's files convention. Used on a folder, it should only escape it, leaving the sub-folders as is.
<!-- Both generates the same api -->
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
β ββ (auth.ts) <!-- {login} -->
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
β ββ (sensitive)
β ββ (auth.ts) <!-- {login} -->
<!-- Now I can add that endpoint, separated from the other -->
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
β ββ (sensitive)
β ββ (auth.ts) <!-- {login} -->
β ββ (whatever.ts) <!-- {notRelatedToLogin} -->
Concern
The structure below will generate two api.member.checkout.*. But I guess it shouldn't be hard to catch during npx convex dev|deploy, like you do when detecting other irregular patterns.
<!-- This throws an Exception at compile|build time -->
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
β ββ (oopsie)
β ββ checkout.ts <!-- {create, notResume} -->
<!-- This one is fine -->
convex
ββ user
β ββ checkout.ts <!-- {create, resume} -->
β ββ crud.ts <!-- {get, create, update, drop} -->
β ββ (not-oopsie)
β ββ checkout.ts <!-- {notCreate, notResume} -->
Conclusion
I really think this could further improve the already great DX around Convex.
Thanks to anyone bringing feedback or support to that proposal.
I think you can use index.ts (eg user/index.ts) to get the API you're looking for.
Try it?
I think you can use
index.ts(eg user/index.ts) to get the API you're looking for. Try it?
I did and unfortunately it doesn't work. It generates api.user.index.*.
But still, even if it worked it wouldn't give as much customization as my proposal. All api.user.* would have to live in index.ts.
ah yeah - you're right. Sorry for the misleading suggestion.
Yeah - this feature idea is good. Either via an index.ts or doing something similar to next.
We'll keep in in mind for going forward. @ianmacartney has been making a list of ideas for a 2.0 of Convex - and something in this space seems reasonable.
I'm glad that you like the proposal. It will be cool to see it in a future version. Maybe even before 2.0? It doesn't seem like a difficult problem to tackle or involving a breaking change.
Anyway, if and whenever you implement this, please consider my version of it instead of the one using index.ts. As mentioned, the latter would solve some cases but leave other drawbacks. If your team takes the time to add the feature, they might as well go for the fully customizable version rather than something that people will eventually complain about again.
Here's the way I think about it:
Supporting index.ts would be convenient, I agree. It looks a bit ugly/verbose, and it's hard to migrate from having users.ts to /users without a breaking change.
I see folder naming as an intent to organize code differently than where you expose endpoints. This is already possible today! You can put code in a lib/checkout/ folder, then re-export them from a top-level /orders.ts. You can decide whether to export it in both locations, or only in one.
Here's a repo I just made to show how to do it: https://github.com/ianmacartney/code-organization-pattern
The things I don't like about /(foo)/[bar]/[[...baz]] and other framework-specific tricks:
- It's not clear where a function is defined.
- There's nothing in typescript that will prevent you from defining the same api path in multiple files
- If you don't currently use Next.js, this behavior is strange and unexpected. I personally am always unsure what each magic filename trick will do. For a website UI it makes more sense, since the user sees the URL and you want it to look pretty. For an internal API, it's just a matter of functionality and making it easy for the developer. Adding
/user/:id/foois a less type-safe way to add parameters. - The lack of adherence to established patterns - /index has been around since the start of the internet. Each new file naming "feature" is framework-specific. Tanstack Start has
/foo/$bar, and others have their own quirks, but none of them afaik share these folder paterns.
I know this probably isn't what you want to hear, but this is my honest take. Hopefully the repo is a good example
I agree that having an index.ts would actually be really convenientβit would cover some edge cases where you donβt want to create a separate sub-path.
As for the rest, Iβm with @ianmacartney: you should try to separate your helper functions from what I call the router folder, where I export the routes.
The idea is to use the router folder strictly to expose your API (and maybe include some controller logic), and move any shared logic or utility functions into a separate helpers folder. I find this architecture pretty clean and maintainable.
Also, it helps separate the core logic of the app from the permissions control, so you only need to manage permissions in a single entry point.
Thanks for the feedback from everyone and sorry for my late response. Here are my thoughts:
- It's not clear where a function is defined.
Ctrl + click would still work and be the most used way to locate a function's definition. Otherwise, my guess is that if you organize your folders with escaped names in a certain way, you'll probably be able to find your way through them as it's your own organization anyway. Whether you're solo or your team decided of a consensus.
- There's nothing in typescript that will prevent you from defining the same api path in multiple files
Just like nothing stopped me from using redis in a mutation, getting an error at runtime telling me to use an action. As I mentionned in the proposal, my guess was that your CLI could catch duplicate api's paths during npx convex dev|deploy. Just like it already does for other wrong patterns.
- If you don't currently use Next.js, this behavior is strange and unexpected. I personally am always unsure what each magic filename trick will do. For a website UI it makes more sense, since the user sees the URL and you want it to look pretty. For an internal API, it's just a matter of functionality and making it easy for the developer. Adding
/user/:id/foois a less type-safe way to add parameters.- The lack of adherence to established patterns - /index has been around since the start of the internet. Each new file naming "feature" is framework-specific. Tanstack Start has
/foo/$bar, and others have their own quirks, but none of them afaik share these folder paterns.
My proposal needs only one magic trick that will escape the folder/file's name. If you were to pick (), no need for _ [] [...] [[...]]. I get why you're reluctant, but Convex itself already introduces new patterns that I had to learn. As how queries and mutations don't run in node, must be deterministic, can't execute js for more than one second, and so on.
This fundamentally changed the way my api was originally built with tRPC - Drizzle - Turso. I've spent a lot of time scratching my head as I was migrating, due to these Convex-specific patterns. I really don't see how introducing an escape pattern could be problematic. Especially that it would be opt-in and not something forced upon you, unlike the ones cited above.
I see folder naming as an intent to organize code differently than where you expose endpoints. This is already possible today! You can put code in a
lib/checkout/folder, then re-export them from a top-level /orders.ts. You can decide whether to export it in both locations, or only in one. Here's a repo I just made to show how to do it: https://github.com/ianmacartney/code-organization-pattern
Supporting index.ts would be convenient, I agree. It looks a bit ugly/verbose, and it's hard to migrate from having users.ts to /users without a breaking change
I know this probably isn't what you want to hear, but this is my honest take. Hopefully the repo is a good example
Unfortunately, the repo is empty. Is that the dev's version of being Rickrolled? lol.
Anyway, I have overlooked the fact that you can define your functions anywhere and then export them from the convex folder. It's probably worth a mention somewhere in your Docs. In addition with support for index.ts, I think it'd get rid of all the drawbacks my proposal aimed to fix.
Thanks again for the feedback!
Oops, just pushed: https://github.com/ianmacartney/code-organization-pattern Thanks for the input!
On Mon, Jun 16, 2025 at 1:32β―AM WKD @.***> wrote:
wkd-kapsule left a comment (get-convex/convex-backend#102) https://github.com/get-convex/convex-backend/issues/102#issuecomment-2975581523
Thanks for the feedback from everyone and sorry for my late response. Here are my thoughts:
- It's not clear where a function is defined.
Ctrl + click would still work and be the most used way to locate a function's definition. Otherwise, my guess is that if you organize your folders with escaped names in a certain way, you'll probably be able to find your way through them as it's your own organization anyway. Whether you're solo or your team decided of a consensus.
- There's nothing in typescript that will prevent you from defining the same api path in multiple files
Just like nothing stopped me from using redis in a mutation, getting an error at runtime telling me to use an action. As I mentionned in the proposal, my guess was that your CLI could catch duplicate api's paths during npx convex dev|deploy. Just like it already does for other wrong patterns.
- If you don't currently use Next.js, this behavior is strange and unexpected. I personally am always unsure what each magic filename trick will do. For a website UI it makes more sense, since the user sees the URL and you want it to look pretty. For an internal API, it's just a matter of functionality and making it easy for the developer. Adding /user/:id/foo is a less type-safe way to add parameters.
- The lack of adherence to established patterns - /index has been around since the start of the internet. Each new file naming "feature" is framework-specific. Tanstack Start has /foo/$bar, and others have their own quirks, but none of them afaik share these folder paterns.
My proposal needs only one magic trick that will escape the folder/file's name. If you were to pick (), no need for _ [] [...] [[...]]. I get why you're reluctant, but Convex itself already introduces new patterns that I had to learn. As how queries and mutations don't run in node, must be deterministic, can't execute js for more than one second, and so on.
This fundamentally changed the way my api was originally built with tRPC - Drizzle - Turso. I've spent a lot of time scratching my head as I was migrating, due to these Convex-specific patterns. I really don't see how introducing an escape pattern could be problematic. Especially that it would be opt-in and not something forced upon you, unlike the ones cited above.
I see folder naming as an intent to organize code differently than where you expose endpoints. This is already possible today! You can put code in a lib/checkout/ folder, then re-export them from a top-level /orders.ts. You can decide whether to export it in both locations, or only in one. Here's a repo I just made to show how to do it: https://github.com/ianmacartney/code-organization-pattern
Supporting index.ts would be convenient, I agree. It looks a bit ugly/verbose, and it's hard to migrate from having users.ts to /users without a breaking change
I know this probably isn't what you want to hear, but this is my honest take. Hopefully the repo is a good example
Unfortunately, the repo is empty. Is that the dev's version of being Rickrolled? lol.
Anyway, I have overlooked the fact that you can define your functions anywhere and then export them from the convex folder. It's probably worth a mention somewhere in your Docs. In addition with support for index.ts, I think it'd get rid of all the drawbacks my proposal aimed to fix. Thanks again for the feedback!
β Reply to this email directly, view it on GitHub https://github.com/get-convex/convex-backend/issues/102#issuecomment-2975581523, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACZQWZGYBP7EG2BQH2ORHL3DZ6I7AVCNFSM6AAAAAB5WBBCRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZVGU4DCNJSGM . You are receiving this because you were mentioned.Message ID: @.***>
Thanks for the update. If I'm correct, your structure creates both api.myFunctions.myMutation and api.baz.buz.myMutation which are the same function. I'm wondering, what is the use case for this? Wouldn't you want to only use the pattern of barApi?
The former is backwards compatible to today's guidance (exporting
everything), and doesn't require a separate export { myMutation } on the
module you may forget. Both should work though
On Mon, Jun 16, 2025 at 1:45β―PM WKD @.***> wrote:
wkd-kapsule left a comment (get-convex/convex-backend#102) https://github.com/get-convex/convex-backend/issues/102#issuecomment-2978077947
Thanks for the update. If I'm correct, your structure creates both api.myFunctions.myMutation and api.baz.buz.myMutation which are the same function. I'm wondering, what is the use case for this? Wouldn't you want to only use the pattern of barApi?
β Reply to this email directly, view it on GitHub https://github.com/get-convex/convex-backend/issues/102#issuecomment-2978077947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACZQW6VQKUKZTHKJPHP2ZL3D4UIJAVCNFSM6AAAAAB5WBBCRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZYGA3TOOJUG4 . You are receiving this because you were mentioned.Message ID: @.***>
Right now Im declaring custom object in api layer on frontend. Would be nice to be able to configure it on convex side
convex/
ββ models/
β ββ user/
β β ββ schema.ts
β β ββ types.ts
β β ββ index.ts // CRUD
β ββ post/
β β ββ index.ts // CRUD
src/
ββ lib/
β ββ convexApi.ts
// convexApi.ts
export const convexApi = {
user: api.models.user.index,
post: api.models.post.index,
};
i support this..keeping backend modular is nightmare with convex.
If there was a way to bypass file-based routing and provide explicit
routers (which could be nested), and you could register a whole file with
something like import * as foo from "./foo.js", then I think we'd end up
in a more powerful place. Where file-based routing is the convenient day-1
routing solution, but once you have a structure you want to enforce you can
build yourself a structure that's concise, where the API doesn't need to
match your directories. Then things like api.foo.bar could be replaced
with fooRouter.bar and sections of the codebase could be more well
contained.
Just a thought, but would that provide what you're looking for?
On Sun, Oct 26, 2025 at 8:03β―AM Jimmy Harika @.***> wrote:
jimmyharika left a comment (get-convex/convex-backend#102) https://github.com/get-convex/convex-backend/issues/102#issuecomment-3448611846
i support this..keeping backend modular is nightmare with convex.5213A44B-5F31-43DC-928A-B67D091DFBBA.jpeg (view on web) https://github.com/user-attachments/assets/6b7eef82-4d48-48cc-98eb-1cf2b1c08332
β Reply to this email directly, view it on GitHub https://github.com/get-convex/convex-backend/issues/102#issuecomment-3448611846, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACZQW7HTIQIZGODP2KGNFD3ZTPDBAVCNFSM6AAAAAB5WBBCRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINBYGYYTCOBUGY . You are receiving this because you were mentioned.Message ID: @.***>