TileDB icon indicating copy to clipboard operation
TileDB copied to clipboard

Bug test for REST array schema evolution

Open jp-dark opened this issue 1 year ago • 7 comments

Add REST test for array schema evolution


TYPE: NO_HISTORY

jp-dark avatar Jul 01 '24 19:07 jp-dark

Thanks a lot @jp-dark for that test, it's really useful! I'll look at the underlying defect as soon as I sort out some other priorities.

@johnkerl @jp-dark would you be temporarily unblocked in this scenario if you use Query v3? As you can see in the gitlab runner we don't have that defect if we use rest.use_refactored_array_open_and_query_submit . I know we have another issue with Qv3 though, and that's the one I am actively working on, but maybe it's worth trying if in this scenario that issue doesn't show up and it can get you running.

ypatia avatar Jul 02 '24 07:07 ypatia

@ypatia my understanding was that query v3 had other issues. Let me check my notes and get back to you.

johnkerl avatar Jul 02 '24 12:07 johnkerl

@ypatia my understanding was that query v3 had other issues. Let me check my notes and get back to you.

yeap, it has this one : https://app.shortcut.com/tiledb-inc/story/48707/502-bad-gateway-on-soma-query-with-rest-use-refactored-array-open-and-query-submit was just wondering if that's the same use case/scenario.

ypatia avatar Jul 02 '24 12:07 ypatia

was just wondering if that's the same use case/scenario.

@ypatia SC 48707 reports a problem with ingesting (writing) data with

        "rest.use_refactored_array_open": True,
        "rest.use_refactored_array_open_and_query_submit": True,

The issue on this PR with querying (reading) data where the schema has been evolved

johnkerl avatar Jul 02 '24 13:07 johnkerl

@ypatia SC 48707 reports a problem with ingesting (writing) data with

Your repro scripts in SC-48707 seem like read queries to me. Am I missing something? I am actively working on the resolution as a prioirity so I'd really like to know more on that issue.

Did you have the change to try this schema evolution scenario with query v3?

ypatia avatar Jul 03 '24 09:07 ypatia

Your repro scripts in SC-48707 seem like read queries to me. Am I missing something?

You're right, sorry for the confusion -- sc-48707 is indeed a read query.

Did you have the change to try this schema evolution scenario with query v3?

@ypatia thank you -- I'll try this right now over morning coffee.

johnkerl avatar Jul 03 '24 11:07 johnkerl

(More conversation in Slack BTW.)

johnkerl avatar Jul 03 '24 14:07 johnkerl