feat: Add datastore mode data transforms
Summary:
This PR adds support for datastore mode data transforms. A new option is provided for the first argument in the save method and code is added that will build the entity proto so that the backend receives requests that will do Datastore Mode data transforms.
API change
Here is an example of a call that the user can now make with the save method. They can now provide transforms that will do data transforms for a commit operation:
const result = await datastore.save({
key: key,
data: {
name: 'test',
p1: 3,
p2: 4,
p3: 5,
a1: [3, 4, 5],
},
transforms: [
{
property: 'p1',
setToServerValue: true,
},
{
property: 'p2',
increment: 4,
},
{
property: 'p3',
maximum: 9,
},
{
property: 'p2',
minimum: 6,
},
{
property: 'a1',
appendMissingElements: [5, 6],
},
{
property: 'a1',
removeAllFromArray: [3],
},
],
});
Notes about the API surface:
maximum,minimum,increment,setToServerValueare encoded togoogle.datastore.v1.IValueso they can representanyvalue sincegoogle.datastore.v1.IValueis an encoded version of any value type. Likewise,appendMissingElementsandremoveAllFromArraycan accept any array ofanyvalues since they are encoded into thegoogle.datastore.v1.IArrayValuetype.- In the proto there is a one to many relationship between a mutation and property transforms. In the save function the user provides a list of entities which translates to a list of mutations for the request so we should have one list of transforms per entity.
Alternatives:
We could support an API where transforms are indexed by the property:
const result = await datastore.save({
key: key,
data: {
name: 'test',
p1: 3,
p2: 4,
p3: 5,
a1: [3, 4, 5],
},
transforms: {
p1: {
setToServerValue: true, // Set transforms
},
p2: {
increment: 4,
}
},
});
Pros:
- Less code
Cons:
- Doesn't match the structure of the proto
- Order can be important for transforms so this gives the user less control
Changes:
src/entity.ts: These are just interface changes. Technically they would affect users that currently supply a transform property, but I doubt any users currently use a transform property. We could remove this file and the PR would still be good.
src/index.ts: The save method is modified to attach propertyTransforms to the entity proto
src/utils/entity/buildPropertyTransforms.ts: This file is added to build the propertyTransforms from the user's transforms array.
system-test/datastore: A test is added that performs a save operation with transforms and ensures the encoded entity proto is correct, the transforms returned are correct and the backend data after the update is correct.
Next Steps:
We should add support for users that wish to do transforms with transactions.
Requests for the reviewer:
- Be sure to take a look at the API and ensure that it is the preferred choice
- Consider the suggestion of removing the
src/entity.tsfile.
LGTM !