directus-sync icon indicating copy to clipboard operation
directus-sync copied to clipboard

Data sync + default data

Open Attacler opened this issue 2 years ago • 4 comments

It would be nice to have an option to sync actual data. For example for API keys or data that always needs to be the same across deployments.

The other suggestion i have is to add default data. This could come in handy in an ecommerce system with VAT rules or shipping options.

The only thing to watch out is relations, you cannot assume that every system has the same ids or relations.

Attacler avatar Nov 14 '23 17:11 Attacler

Thank you for the suggestions.

I believe we could combine these suggestions into a single feature. I'm curious about how we might define which collections and fields need to be synchronized. Using CLI options or Environment Variables might be verbose.

I also think we should add a --no-seed option to avoid inserting "seed" data during the push command. In some cases, we would want to update the Directus configurations without inserting this "seed" data. Same thing for the pull command

Finally, regarding the IDs issue, it's true that if we have relations to collections that are not being synchronized, we cannot ensure the correspondence of IDs. It might be the user's responsibility to not synchronize fields containing unsynchronized relations. Initially, we could implement this without considering relations, and then improve the feature to take into account the relations.

EdouardDem avatar Nov 15 '23 14:11 EdouardDem

Would love this as well! Is there another method to sync the data at this time?

notflip avatar Feb 11 '24 18:02 notflip

Excited to see this being worked on.

Besides smaller data like shipping options, it would unlock some exciting workflows if the implementation was robust enough to handle larger collection, e.g. a whole product catalog.

I'm sure at some point this could get a bit unwieldy to check in too much data to version control, but it'd be interesting to see how far it could scale as a deployment method. As synced collections grow, maybe it would be helpful to split records out by file considering memory usage in general, and difficulty with larger file size diffs in Git.

It might be the user's responsibility to not synchronize fields containing unsynchronized relations. Initially, we could implement this without considering relations, and then improve the feature to take into account the relations.

I noticed that directus-schema-sync supports syncing custom collections, but its docs mention it's up to you to sync any data with relations in the right order. Leaving it up to the user to control this seems like a reasonable simple strategy.

wuservices avatar May 21 '24 22:05 wuservices

@EdouardDem how about putting this command in another subcommand like npx directus-sync seed ?

vwasteels avatar Jul 31 '24 13:07 vwasteels

Given that this issue is closed, I take it synchronizing (as opposed to seeding) data is not planned?

rafalkrupinski avatar Apr 22 '25 16:04 rafalkrupinski