orama
orama copied to clipboard
Alert before inserting unserializables types into a schemaless instance
Once we get #174 merged, we will be able to insert any kind of valid JSON object in Lyra. That also means that the following could be a valid document:
{
"foo": "bar",
"baz": () => {},
"quick": 10n
}
In the above example, we can clearly see how it will be impossible to serialize functions and BigInt data without applying any kind of normalization.
My proposal would be to either:
- Spawn an alert when inserting non-serializable data (functions, maps, sets, bigints, etc.)
- Throw an error when inserting non-serializable data
I'd personally go with option 2, but I am open for comments
Hei @micheleriva ! I just wrote a possible solution, updating the recursiveCheckDocSchema
function to take advantage of the calculus that's already there. I checked "bigint", "symbol", "function", "Array", "Map", "WeakMap", "Set".
This is the updated implementation:
function recursiveCheckDocSchema<S extends PropertiesSchema>(
newDoc: ResolveSchema<S>,
schema: PropertiesSchema,
): { valid: boolean, key?: string } {
for (const key in newDoc) {
const propType = typeof newDoc[key];
if (!(key in schema)) {
if(["bigint", "symbol", "function"].indexOf(propType) >= 0) {
return { valid: false, key };
}
if(propType === "object") {
if (["Array", "Map", "WeakMap", "Set"].indexOf(newDoc[key].constructor.name) >= 0) {
return { valid: false, key };
}
return recursiveCheckDocSchema(newDoc[key] as ResolveSchema<S>, schema);
}
continue;
}
if (propType === "object") {
return recursiveCheckDocSchema(newDoc[key] as ResolveSchema<S>, schema);
} else if (propType !== schema[key]) {
return { valid: false };
}
}
return { valid: true };
}
I don't like 100% a couple of things:
- checking in the same function both the schema validity and the unserializables type (but maybe it is worth using the same loop to re-do the calculus)
- the function name is not so clear anymore
- I had to update the return values to have the key and throw a different error, like:
export function INVALID_DOCUMENT(key: string): string {
return `${key} field is not serializable.`
}
The only doubt is how to catch also custom class instances. What do you think?
Hello there @stearm
This works quite well (besides a previous bug about nested schema checks that I'm fixing with this), I was wondering about the usage of Object.getPrototypeOf(newDoc[key])
and the usage of newDoc[key] instanceof X
. This could be tricky in some environments like edge workers and older browsers runtimes
What do you guys think about it?
ps. I'm including part of your snippet in my PR #185 as you can see here as first implementation, hope you don't mind @stearm 🙏