Please provide more flexible gql querying

[jsoneaday]

Currently all queries are effectively AND queries. We need to have things like OR, Contains, Greater than or Less Than, Starts With, Ends With, Not Equal, Ranges, etc.

This is hard but for some workflows really necessary. Perhaps start with Contains and Greater Than and Less Than?

[hlolli]

I'm happy to help design individual query, but there's a limitation inherent with OR and NOT_EQ queries which I can't fix. There are good talks and articles describing why this isn't a feature supported in big-data databases and whey they are inherently slow. There should otoh always be a way to query the data in alternative kind of fashion that you get what you are looking for. Ranges are supported (height min height max in blocks query as example), contains is also possible ex. providing list of ids. But each way of querying requires enormous amount of time in design.

In any case, some feature could be missing which we can support from the list of contrains you mentioned. If we find the highest priority queries we need to support, we'll work on support for them one by one, then I think we are on a good track, knowing that still, it's a lot of work making fast queries with huge amount of data (which is getting exponentially bigger these days).

[jsoneaday]

I understand your concern, but even adding one of those options would help

Thanks

Sent from my iPhone

On Aug 21, 2021, at 12:55 PM, Hlöðver Sigurðsson @.***> wrote:



I'm happy to help design individual query, but there's a limitation inherent with OR and NOT_EQ queries which I can't fix. There are good talks and articles describing why this isn't a feature supported in big-data databases and whey they are inherently slow. There should otoh always be a way to query the data in alternative kind of fashion that you get what you are looking for. Ranges are supported (height min height max in blocks query as example), contains is also possible ex. providing list of ids. But each way of querying requires enormous amount of time in design.

In any case, some feature could be missing which we can support from the list of contrains you mentioned. If we find the highest priority queries we need to support on work on support for them one by one, then I think we are on a good track, knowing that still, it's a lot of work making fast queries with huge amount of data (which is getting exponentially bigger these days).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FArweaveTeam%2Fvartex%2Fissues%2F25%23issuecomment-903144413&data=04%7C01%7C%7C1f00f9914acb4e7dfc2408d964c468e0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637651617001071050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6lXZFp02MudGBupH1Xzr5yQTdq9NSE9yRGH77Hi58Io%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAALIC552PZIVXLIE35UUZM3T57K6DANCNFSM5CR6D7WQ&data=04%7C01%7C%7C1f00f9914acb4e7dfc2408d964c468e0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637651617001071050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nOJWdQ8VkpGNDkKDDk6jPFctL%2FzkNAeZDSoeLVp4rw4%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C1f00f9914acb4e7dfc2408d964c468e0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637651617001081044%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Pg7NMaWMAOSLmg1TpJDzzFCpV19d2VUkLFDwqZLl4tg%3D&reserved=0 or Androidhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26utm_campaign%3Dnotification-email&data=04%7C01%7C%7C1f00f9914acb4e7dfc2408d964c468e0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637651617001081044%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0L6BFjAeXgpTp6mFMI49nv2ZWGW%2FZkVdqD%2BDB0egv60%3D&reserved=0.