redash icon indicating copy to clipboard operation
redash copied to clipboard

bigquery load schema diff locations ignore

Open Lee2532 opened this issue 10 months ago • 10 comments

What type of PR is this?

  • [x] Bug Fix

Description

An error occurs when retrieving the BigQuery schema if there are datasets located in different regions. This update ensures that the correct region is specified to prevent such issues.

How is this tested?

  • [x] Manually

if dataset location is different then set location : asia-northeast3 test-dataset-location : US Not fount: dataset test-data-set was not found in location asia-northeast3

Lee2532 avatar Jan 17 '25 08:01 Lee2532

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 64.00%. Comparing base (4ee53a9) to head (e83fa23). Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
redash/query_runner/big_query.py 0.00% 3 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7289      +/-   ##
==========================================
- Coverage   64.01%   64.00%   -0.02%     
==========================================
  Files         163      163              
  Lines       13410    13413       +3     
  Branches     1905     1906       +1     
==========================================
  Hits         8585     8585              
- Misses       4490     4493       +3     
  Partials      335      335              
Files with missing lines Coverage Δ
redash/query_runner/big_query.py 23.56% <0.00%> (-0.38%) :arrow_down:

codecov[bot] avatar Jan 17 '25 08:01 codecov[bot]

LGTM!

yoshiokatsuneo avatar Jan 21 '25 05:01 yoshiokatsuneo

@eradman Hi, It seems that an error is occurring in the coverage. Is it necessary to add additional tests?

Lee2532 avatar Jan 23 '25 11:01 Lee2532

@eradman Hi, It seems that an error is occurring in the coverage. Is it necessary to add additional tests?

Yes, we should test for this condition. Try adding a test to tests/query_runner/test_bigquery.py

eradman avatar Jan 23 '25 14:01 eradman

@eradman Hi, It seems that an error is occurring in the coverage. Is it necessary to add additional tests?

Yes, we should test for this condition. Try adding a test to tests/query_runner/test_bigquery.py

@eradman

It is nice to have 100% code coverage. But at least for query runners, current code coverage is quite low, and it is not so trivial to add tests for whole query runners. So, I feel it is not so practical to request tests only for new lines for now.

yoshiokatsuneo avatar Jan 23 '25 16:01 yoshiokatsuneo

@eradman Hi, It seems that an error is occurring in the coverage. Is it necessary to add additional tests?

Yes, we should test for this condition. Try adding a test to tests/query_runner/test_bigquery.py

@eradman

It is nice to have 100% code coverage. But at least for query runners, current code coverage is quite low, and it is not so trivial to add tests for whole query runners. So, I feel it is not so practical to request tests only for new lines for now.

I'm of the same opinion

Lee2532 avatar Jan 24 '25 09:01 Lee2532

@Lee2532 I noticed that this prevent loading any dataset when "Processing Location" is not set. I think it is better to check location only when the location configuration was set.

yoshiokatsuneo avatar Feb 12 '25 05:02 yoshiokatsuneo

@eradman hi, can you check this workflow? thanks

Lee2532 avatar Feb 12 '25 12:02 Lee2532

@eradman hi, can you check this workflow? thanks

Workflow started

eradman avatar Feb 12 '25 13:02 eradman

@eradman Hi, I think all the tests have passed, it can merge?

Lee2532 avatar Feb 14 '25 10:02 Lee2532

@eradman

I think this PR is good and helpful, and we can approve and merge it. Just to make sure, do you have any concern ?

Thank you !

yoshiokatsuneo avatar Jul 19 '25 02:07 yoshiokatsuneo

I tested and confirmed that the PR fixes the problem on my environment.

yoshiokatsuneo avatar Jul 19 '25 15:07 yoshiokatsuneo