dynamodb-toolbox
dynamodb-toolbox copied to clipboard
jeremydaly
Computed properties primary key for GraphQL
Hi,
Thanks for the great library.
So I am building a GraphQL API and using your awesome library to do it. On the frontend I am using Apollo client, which likes to have an ID property on objects so it can cache them on the frontend.
Dynamo likes composite primary keys, and I cannot find a great way to express this for GraphQL so I figured I would just create the primary key ID and add it to each GraphQL object. Just Partition Key + Sort Key = Primary Key
Ideally I don't want to save this in the DB as I can easily compute it every time. Is there a good way to do this currently? Should I just save it? If not would you be open to having computed fields?, kinda like defaults... it's just you don't save them.
attributes: {
id: {
// something like this
computed: (data) => `${data.PK}${data.SK}`,
},
uid: {
partitionKey: true,
prefix: UID_PREFIX,
},
sk: {
sortKey: true,
hidden: true,
dependsOn: ['programSubscriptionId', 'startDateTime'],
default: createProgramSubscriptionSK,
},
....
Similar-ish to this question
https://github.com/jeremydaly/dynamodb-toolbox/issues/32
Instead of returning the entity, map the entity into a business object that calculates that for you.
class BusinessObject {
constructor(entity) {
...
this.id = '${entity.hashKey}#${entity.rangeKey}
}
}
const entity = Entity.get(..);
const business = new BusinessObject(entity);
return business;
@darbio
Thanks, for you quick reply.
That is a workable solution. One issue however is when I return an entity the SK and PK are potentially hidden, so I cannot just pass the entity into the BusinessObject constructor, unless I unhide them, then omit them from the business object.
Totally do-able but I feel like the point of mapping the entity attributes with dynamodb-toolbox is already to create business objects (Entities) and have them map to an underlying different data model. That's why it would be nice to have the computed field on the entity map, as it has all the underlying attributes. Thoughts?
It depends upon your definition of entity.
I tend to define entities in my projects as things that represent persistence.
Business logic (such as computing an id that only matters in the business domain) does not belong on the entity - instead it belongs on the business model (or domain object).
I don't think that this library is supposed to be giving you the business layer - it's a tool to help you manage cramming your entities into a single table using dynamodb.
What happens when your entity model doesn't match your business model? With no separation between the two, you are going to be forcing the business model to bend to the whims and needs of the data persistence model. For smaller projects this probably isn't an issue, however once your project gets larger it's gonna bite you on the backside.
FWIW I use typestack class transformer (and validator) to convert (map) between my entities and business (domain) objects.
@darbio
This is almost a workable solution.
Upon an initial attempt at implementing however, PK is a prefix and is mapped, I would want the raw PK, SK values to create the primary key, so including the prefix. I can totally manually include the prefix but I really like trying to keep it in one place.
So ideally I would still need some way to get at raw PK, SK and have my entity map correctly, I mean currently I think sans prefixes it would still be unique but potentially you could have two item with a PK SK combination where the attributes without their prefixes would not be unique. You know for sure the PK SK combination with prefixes would be unique as Dynamo will not let you save a duplicate Primary Key.
So yeah business objects could work great if there was an easy way for me to get the raw PK SK, along side the entity:
await this.sessionLog.get(params, {rawAttributes: ['PK', 'SK']})
But then it would still be really nice to have computed properties on the entity.
Side note: I will checkout class transform. Thanks for that.
Again thanks for your help. I noticed you are in Sydney maybe I can buy you coffee some time :)
Hey,
So I can totally, do something like this:
const { Item: log } = await this.sessionLog.get(params, { parse: false });
parsedLog = this.sessionLog.parse(log)
bo = BusinessObject(parsed, PK, SK)
However it would still be nice if dynamodb-toolbox made it easier for me.
@vespertilian, I'd love to understand this use case a bit better. The prefixing/suffixing capabilities are a way to help abstract the need for managing the raw PKs and SKs, but it looks like you just need a way to calculate the PK/SK to do a look up? So, you'd still need to parse that generated id field on a cache miss and pass it into the library for the look up, correct?
@jeremydaly
Thanks for your quick reply.
So my use case is that I am building an app with a backend API in GraphQL.
The problem that I am trying to solve is that the frontend library I am using to consume the GraphQL Api (Apollo) likes to have id's on all the object types as it caches them by type and id.
I figured it's easy enough to add an ID as the primary key for every entity I have, which will always be unique is just PK + SK = Primary Key.
// ┌───────────────────────────────────────────────┬──────────────────────────────────────────────────────┐
// │ Primary Key │ │
// ├────────────┬──────────────────────────────────┤ Attributes │
// │ PK │ SK │ │
// ├────────────┼──────────────────────────────────┼────────────────────────────┬────────────┬────────────┤
// │ │ │ type │ programId │ ...other │
// │ │ ├────────────────────────────┼────────────┼────────────┤
// │ │ PSUBID#psub-id__date_time │ │ │ │
// │ │ │ SESSION_LOG │ pid-1 │ foo │
// │ │ │ │ │ │
// │ ├──────────────────────────────────┼────────────────────────────┼────────────┼────────────┤
// │ │ │ type │ programId │ ...other │
// │ │ ├────────────────────────────┼────────────┼────────────┤
// │ UID#user_1 │ PSUBID#psub-id__date_time │ │ │ │
// │ │ │ SESSION_LOG │ pid-1 │ foo │
// │ │ │ │ │ │
// │ ├──────────────────────────────────┼────────────────────────────┼────────────┼────────────┤
// │ │ │ type │ programId │ ...other │
// │ │ ├────────────────────────────┼────────────┼────────────┤
// │ │ PSUBID#psub-id__9999-01-01... │ │ │ │
// │ │ │ USER_PROGRAM_SUBSCRIPTION │ pid-1 │ foo │
// │ │ │ │ │ │
// └────────────┴──────────────────────────────────┴────────────────────────────┴────────────┴────────────┘
A specific use case: Session Log (simplified)
{
uid: string // PK sample: UID#vesperilian_id
programSubscriptionId: string // SK sample: PSUBID#program-sub-id__2021-01-19T02:58:49.063Z
... otherParams
}
So Primary key should = UID#vesperilian_id___PSUBID#program-sub-id__2021-01-19T02:58:49.063Z I cannot just use the PUSBID or the UID as they may not be unique for this entity type. Now i can just add this to my GraphQL object type.
So it would be great if there was a way to compute the underlying primary key when using a composite key (prefixes and all)
My initial thought was just something like this.
id: {
// something like this
computed: (data) => `${data.PK}${data.SK}`,
},
@darbio Here is my current full implementation for my session log, this is all new and influx. @jeremydaly With @darbio help above (thanks) I have managed to workout a solution, but it still feels a little more awkward than I would like. Currently I am just hooking into unparsed query, grabbing the PK, SK and then parsing it and adding the result. I created a function for this as I will likely use this with all my other entities that need the ID.
Also note that when using the put command with an object I return the newly created object with the ID as it seems to be best practice in a GraphQL mutation to return the recently created object. Would be nice if I could just add an option to the entity config for this assuming we add the computed field.
@Injectable()
export class SessionLogService {
sessionLog: Entity<SessionLogData>;
constructor(private readonly dynamoTableService: DynamoTablesService) {
this.sessionLog = new Entity<SessionLogData>({
name: SESSION_LOG_ENTITY_NAME,
attributes: {
uid: {
partitionKey: true,
prefix: UID_PREFIX,
},
sk: {
sortKey: true,
hidden: true,
dependsOn: ['programSubscriptionId', 'startDateTime'],
default: createProgramSubscriptionSK,
},
programSubscriptionId: { type: 'string', required: true },
startDateTime: { type: 'string', required: true },
endDateTime: { type: 'string' },
duration: { type: 'string' },
currentProgramCustomVarState: { type: 'map' },
activityLogs: { type: 'list' },
},
table: dynamoTableService.carabiner,
});
}
async getUserSessionLog(params: {
uid: string;
programSubscriptionId: string;
startDateTime: string;
}) {
const { Item: rawLog } = await this.sessionLog.get(params, {
parse: false,
});
return returnParsedItemWithPrimaryKey(this.sessionLog)(rawLog);
}
async getLatestUserSessionLogs({
limit,
programSubscriptionId,
uid,
}: LatestUserSessionLogsParams): Promise<SessionLogData[]> {
const pk = `${UID_PREFIX}${uid}`;
const programSubEndSK = createProgramSubscriptionEndSK({
programSubscriptionId,
});
const { Items: rawLogs } = await this.dynamoTableService.carabiner.query(
pk,
{
reverse: true,
limit,
lt: programSubEndSK,
parse: false,
}
);
return rawLogs.map(returnParsedItemWithPrimaryKey(this.sessionLog));
}
async logActivity({
uid,
startDateTime,
programSubscriptionId,
activityLogs,
}: LogActivityData): Promise<SessionLogData> {
const sortedLogs = sortBy(prop('logDateTime'), activityLogs);
const sessionLogParams = {
uid,
programSubscriptionId: programSubscriptionId,
startDateTime,
activityLogs: sortedLogs,
};
const putParams = this.sessionLog.putParams(sessionLogParams);
await this.dynamoTableService.dynamo.put(putParams).promise();
return returnParsedPutParamsWithPrimaryKey(this.sessionLog)(putParams);
}
}
type SchemaType =
| string
| number
| boolean
| {
[key: string]: SchemaType;
}
| SchemaType[];
type EntityType<T> = { [key in keyof T]: SchemaType };
function returnParsedPutParamsWithPrimaryKey<T extends EntityType<T>>(
entity: Entity<T>
) {
return (putParams: { Item: any }) => {
const { Item: rawItem } = putParams;
return returnParsedItemWithPrimaryKey(entity)(rawItem);
};
}
function returnParsedItemWithPrimaryKey<T extends EntityType<T>>(
entity: Entity<T>
) {
return (rawItem) => {
const parsedItem = entity.parse(rawItem) as T;
return {
id: createIdFromPKSK(rawItem),
...parsedItem,
};
};
}
function createIdFromPKSK({ PK, SK }: { PK: string; SK: string }) {
return `${PK}___${SK}`;
}
@jeremydaly Also thanks for the library, it's been so helpful. Email me your Paypal [email protected] and I will send you a coffee or beer (your choice).
@vespertilian where is createProgramSubscriptionSK defined? Can you reuse that?
@darbio
Yes, USER_PROGRAM_SUBSCRIPTION is below the latest session logs in my single table design so you can read backwards and get the program subscription entity and then the latest logs with one read, well that's the current plan anyway. I like tying them together so if I am ever looking at them I trace the code back and realise that it needs to be the same.
import { END_DATE, PSUBID_PREFIX } from './entity-constants';
export function createProgramSubscriptionSK(data: {
programSubscriptionId: string;
startDateTime: string;
}): string {
return `${PSUBID_PREFIX}${data.programSubscriptionId}__${data.startDateTime}`;
}
export function createProgramSubscriptionEndSK({
programSubscriptionId,
}: {
programSubscriptionId;
}): string {
return createProgramSubscriptionSK({
programSubscriptionId,
startDateTime: END_DATE,
});
}
The end date is just the year 9999
export const END_DATE = '9999-01-01T12:59:59.000Z';