Production Optimizations & Scalability - Feature Requests

[shadowcodex]

Is your feature request related to a problem? Please describe.

Using Prisma in a production environment results in unexpected issues. I will outline several of the issues that my team & myself found while implementing Prisma from local development, to production, and to support of Prisma in production.

In my honest opinion, without these items Prisma cannot grow to solve the needs of more complex data projects.

Describe the solution you'd like

Separation of concerns on Prisma Layer for reading and writing.

Currently when you setup a Prisma Cluster (Which there is lack of documentation of), you give it a single endpoint for your database layer.

This single endpoint must allow both reading and writing. This causes an issue when it comes to scaling the database appropriately. Since the same endpoint is required to have writing privileges you can only have a single database instance behind that endpoint. This means that as heavy load comes through your application for reading, it can negatively affect Prisma's ability to write to the database. The only way to scale the database is by giving the database more hardware and vertically scaling it.

However there is a better way.

If Prisma allowed either set individual endpoints for reading and writing or allow Prisma to run against a read only endpoint then one endpoint can have the writing instance of the database, and the second endpoint can have a horizontally scaling database reading instances.

The root of the issue here is that databases do not support having more than one write instance, but do support having more than one reader instances. This means that to scale the database cheaper and rather infinitely for read operations you can spin up more readers behind a load balancer. In a lot of scenarios you will run out of hardware by just increasing single instance capacity.

Separation of Concerns of Secret Key Access

Along the same lines, not every application that connects to Prisma should have the ability to write to Prisma. Currently to connect you share a single secret key. For instance if you were building a read only API it has no need to know a secret key that has the ability to write. Especially if the API is open to the public and thus creates an attack vector. Prisma should support creating multiple secret keys with defined access to data models and rights to insert, update, and read given explicitly for the types in the data model.

Limit of in queue items on a single Prisma instance is 1000 and not configurable

When writing or reading to Prisma, items are added to a stack or queue. The limit of the number of items allowed in this is 1000.

We have very memory heavy machines, and thus we could store more items in ram for the queue. However, this option is not configurable and is hard coded. In order to allow more flexibility this option should be parameterized on Prisma Startup or via a config file.

Addition of optimizations

If you know you are going to be using an item in a type a lot as a filter, you should have the ability to pass configs into a datamodel that allow Prisma to create advanced indexes on data within a relational database. While adding unique does create a unique index, some index or complex indexes have to be made manually to the database after Prisma creates the data schemas. Having the ability to define these in the datamodel would be wonderful.

Support of guaranteed Subscriptions

The subscription system in Prisma works in small capacities. However when presented with production level loads, fast data base insertion, and scaled user presence, the system does not deliver subscription data correctly. A lot of insert and modification subscriptions do not fire when under load, and this is because of the way the Subscription service is architected within Prisma. The service should queue and store actions and guarantee delivery to connected clients.

---

I think Prisma is moving and working towards the right goal, and that in the future as you grow and add support to items like above you will have a more robust platform. These are just items that got in our way when developing, and I hope that this feedback helps you find items to make Prisma better, stronger, and more widespread.

[scottyg2334]

Has there been any improvement to PRISMA to accommodate these concerns? Wondering, because I'm building a large scale project with many Users and Data. I have been intending on using PRISMA for this project, but this post has me definitely second guessing that thought..

[peterrogov]

Since prisma v1 is in maintenance mode it is unlikely that any such modification will take place in this branch. It makes more sense to consider prisma v2 for any new project.