backend icon indicating copy to clipboard operation
backend copied to clipboard

Log many audit changes within a request

Open htuerker opened this issue 1 year ago • 0 comments

Problem:

CSV/Airtable data imports trigger lots of audit changes which causes timeouts due to concurrent requests. (fails at when 100+ concurrent requests with minimal setup)

Possible solution:

I'd suggest extending this endpoint to process an array of audit changes to fix failing requests.

The front-end consumer is here:

https://github.com/rowyio/rowy/blob/a35afa96199473ce9e73e30f41a815ed35f895cc/src/sources/TableSourceFirestore/useAuditChange.ts#L43-L64

We can make requests with "audits" as an array here. The only drawback here is we have to change the current usage(only 4-5 places)

In client:

setAuditChange(
      () =>
        (
          audits: {
            type: "ADD_ROW" | "UPDATE_CELL" | "DELETE_ROW";
            rowId: string;
            data?: { updatedField?: string };
          }[]
        ) =>
          rowyRun({
            route: runRoutes.auditChange,
            body: {
              rowyUser: rowyUser(currentUser!),
              audits: audits.map((audit) => ({
                type: audit.type,
                ref: {
                  rowPath: tableSettings.collection,
                  rowId: audit.rowId,
                  tableId: tableSettings.id,
                  collectionPath: tableSettings.collection,
                },
                data: audit.data,
              })),
            },
          }).catch(console.log)
    );

Here we can write logs iteratively.

const { audits } = req.body;
if(!Array.isArray(audits)) {
    throw new Error("400: Bad Request");
  }
return Promise.all(audits.map(audit => log.write(log.entry(metadata, audit))))

htuerker avatar Aug 18 '22 18:08 htuerker