Question about Duplicate Index error

[dror-weiss]

I thought I could use the same index with a scope, on the same entity, but I was getting the "duplicate index error".

That's fine and a good error when there is no scope, but is this still valid when the indexes have different scopes?

If the scope is set to prefix the pk then, in my opinion, we should be allowed to proceed with same indexes with a different scope.

Example:

// Table
const organization = new Entity({
  model: {
    entity: "organization",
    service: "taskapp",
    version: "1"
  },
  attributes: {
    organizationId: {
      type: "string"
    },
  },
  indexes: {
    byOrg: {
      scope: "org", // <--- Scope is set to unique value "org"
      index: 'gsi1', 
      pk: {
        field: "gsi1-pk",
        composite: []
      },
      sk: {
        field: "gsi1-sk",
        composite: ["organizationId"]
      }
    },
    bySomethingElse: {
      scope: "somethingelse", // <--- Scope is set to another unique value
      index: 'gsi1',
      pk: {
        field: "gsi1-pk",
        ...
      },
      sk: {
        field: "gsi1-sk",
        ...
      }
    }
  }
}, { table: "your_table_name" });

// Invoke
organization.query.byOrg(...).go();
organization.query.bySomethingElse(...).go();

[tywalch]

Hi @dror-weiss 👋

You're correct that using scope affects the pk string saved to DynamoDB. It adds the string "org" or "somethingelse" as a postfix onto the value of the pk field. This is why there can't be different definitions for the same index field: one field, one value, and therefore one definition. Here's a playground link using a slightly altered version of your model above; if you look at the parameter values for gsi1-pk, you can see how scope impacts the value saved.

Let me know if this helps, and/or if you'd like to discuss more 👍

[dror-weiss]

Hey @tywalch ,

I appreciate your quick response!

I may have misunderstood you? The scope is used before the pk composite, not after. I hope the following examples will clear up my question.

This playground example is valid, but I need something similar: same index different scopes.

In other words, why is the following example considered invalid, even though they have different scopes?

[tywalch]

To put it simply (and using your second example),

1. The string you give scope gets added to the string value for the field gsi1-pk; That's how scope is implemented under the hood.
2. There can be only one value for the field gsi1-pk, but your model has two conflicting definitions for what the field should be; one with scope org and the other with scope somethingelse.

[dror-weiss]

I got it, but I'm asking why "there can be only one value for the field gsi-pk"?

Right now you use the index name to "know" what to append to the value of ExpressionAttributeValues[":pk"]. If you extend that, you can unlock better resource management of the table and reduce storage.

Kind of a "collection" for partition key (maybe call it "scopes").

[tywalch]

It sounds like I'm the one missing something here 😅 could you demonstrate what you're hoping to do? Specifically, could you write the parameters you're hoping Electro would create in the case you're describing?