Design: Standalone mechanism to estimate maincloud energy usage

[cloutiertyler]

We need some way for users of standalone to be able to estimate the energy usage of Maincloud energy. The calculation used by Maincloud is given by the following:

export function computeEnergySpendFromMetrics(metric: {
    bytesScanned: bigint;
    rowsScanned: bigint;
    bytesWritten: bigint;
    indexSeeks: bigint;
    reducerFuelUsed: bigint;
    rowsInserted: bigint;
    rowsDeleted: bigint;
    rowsUpdated: bigint;
    indexBytesWritten: bigint;
    indexRowsInserted: bigint;
    indexRowsDeleted: bigint;
    indexRowsUpdated: bigint;
    bytesSentToClients: bigint;
    wasmMemoryByteSeconds: bigint;
    dataSizeTableNumRowSeconds: bigint;
    dataSizeTableBytesUsedByRowSeconds: bigint;
    dataSizeTableNumRowsInIndexSeconds: bigint;
    dataSizeTableBytesUsedByIndexKeySeconds: bigint;
    dataSizeBlobStoreNumBlobSeconds: bigint;
    dataSizeBlobStoreBytesUsedByBlobSeconds: bigint;
}): bigint {
    const {
        bytesScanned,
        rowsScanned,
        bytesWritten,
        indexSeeks,
        reducerFuelUsed,
        rowsInserted,
        rowsDeleted,
        rowsUpdated,
        indexBytesWritten,
        indexRowsInserted,
        indexRowsDeleted,
        indexRowsUpdated,
        bytesSentToClients,
        wasmMemoryByteSeconds,
        dataSizeTableNumRowSeconds,
        dataSizeTableBytesUsedByRowSeconds,
        dataSizeTableNumRowsInIndexSeconds,
        dataSizeTableBytesUsedByIndexKeySeconds,
        dataSizeBlobStoreNumBlobSeconds,
        dataSizeBlobStoreBytesUsedByBlobSeconds,
    } = metric;
    return (
        BigInt(bytesScanned) * energyCostMap.bytesScanned +
        BigInt(rowsScanned) * energyCostMap.rowsScanned +
        BigInt(bytesWritten) * energyCostMap.bytesWritten +
        BigInt(indexSeeks) * energyCostMap.indexSeeks +
        BigInt(reducerFuelUsed) * energyCostMap.reducerFuelUsed +
        BigInt(rowsInserted) * energyCostMap.rowsInserted +
        BigInt(rowsDeleted) * energyCostMap.rowsDeleted +
        BigInt(rowsUpdated) * energyCostMap.rowsUpdated +
        BigInt(indexBytesWritten) * energyCostMap.indexBytesWritten +
        BigInt(indexRowsInserted) * energyCostMap.indexRowsInserted +
        BigInt(indexRowsDeleted) * energyCostMap.indexRowsDeleted +
        BigInt(indexRowsUpdated) * energyCostMap.indexRowsUpdated +
        BigInt(bytesSentToClients) * energyCostMap.bytesSentToClients +
        BigInt(wasmMemoryByteSeconds) * energyCostMap.wasmMemoryByteSeconds +
        BigInt(dataSizeTableNumRowSeconds) * energyCostMap.dataSizeTableNumRowSeconds +
        BigInt(dataSizeTableBytesUsedByRowSeconds) *
            energyCostMap.dataSizeTableBytesUsedByRowSeconds +
        BigInt(dataSizeTableNumRowsInIndexSeconds) *
            energyCostMap.dataSizeTableNumRowsInIndexSeconds +
        BigInt(dataSizeTableBytesUsedByIndexKeySeconds) *
            energyCostMap.dataSizeTableBytesUsedByIndexKeySeconds +
        BigInt(dataSizeBlobStoreNumBlobSeconds) * energyCostMap.dataSizeBlobStoreNumBlobSeconds +
        BigInt(dataSizeBlobStoreBytesUsedByBlobSeconds) *
            energyCostMap.dataSizeBlobStoreBytesUsedByBlobSeconds
    );
}

export const computeEnergySpendFromMetric = (value: bigint, metric: Keys) => {
    return value * energyCostMap[metric];
};

export const energyCostMap = {
    bytesScanned: 400n,
    rowsScanned: 0n,
    bytesWritten: 1_000n,
    indexSeeks: 4_000n,
    reducerFuelUsed: 4n,
    rowsInserted: 0n,
    rowsDeleted: 0n,
    rowsUpdated: 0n,
    indexBytesWritten: 20_000n,
    indexRowsInserted: 0n,
    indexRowsDeleted: 0n,
    indexRowsUpdated: 0n,
    bytesSentToClients: 1_200_000n,
    wasmMemoryByteSeconds: 100n,
    dataSizeTableNumRowSeconds: 1_000n,
    dataSizeTableBytesUsedByRowSeconds: 10n,
    dataSizeTableNumRowsInIndexSeconds: 1_000n,
    dataSizeTableBytesUsedByIndexKeySeconds: 10n,
    dataSizeBlobStoreNumBlobSeconds: 0n,
    dataSizeBlobStoreBytesUsedByBlobSeconds: 10n,
} as const;

Notably xxxSeconds metrics are guage metrics that have been integrated over a time period. In the case of maincloud this timeframe is approximately one minute.

[bfops]

Needs a design

[bfops]

If we have a free tier, people can figure this out by launching their module and getting the breakdown via the website/CLI.

[bfops]

soft recommend deferring this until after we have done a free tier design

[bfops]

We're not married to this being on standalone specifically, we just need some way of estimating energy usage in maincloud. this could be using standalone, but could also be a "usage calculator" on your module page, or something else.