ibis icon indicating copy to clipboard operation
ibis copied to clipboard

feat: Consistent tablenotfound exception across backends

Open ssabdb opened this issue 1 year ago • 8 comments

Is your feature request related to a problem?

Different backends expose tables not being found in different ways.

For example,

connection = ibis.connect("pandas://")
connection.table("missing-table")
KeyError: 'missing-table'
connection = ibis.connect("bigquery://....")
connection.table("missing-table")
google.api_core.exceptions.NotFound: '...'

What is the motivation behind your request?

What I'd love to be able to do is to catch a consistent error in the handling code which is agnostic of the underlying backend, e.g.

def my_function(connection):
    try:
        connection.table("missing-table")
    except IbisTableNotFound:
        <handle cleanly>

Describe the solution you'd like

.

What version of ibis are you running?

9.0.0

What backend(s) are you using, if any?

DuckDB, Bigquery

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

ssabdb avatar Jun 28 '24 16:06 ssabdb

+1, definitely a good idea!

cpcloud avatar Jun 30 '24 18:06 cpcloud

This was added on the 9.1.0 version: Screenshot 2024-07-20 at 02 32 45

relmaazouz avatar Jul 20 '24 00:07 relmaazouz

It raises IbisError on postgres, but:

  • IbisError seems like a generic exception, not an exception which is explicit about TableNotFound. A table not existing is sufficiently generic and common across backends that (in my opinion) it should be part of the API.
  • This definitely isn't the case with the bigquery backend even in 10.x, when released. I don't think that's an error with Bigquery, I think that's inconsistency across backends on how to catch missing tables.

image

ssabdb avatar Jul 22 '24 17:07 ssabdb

yeah, this isn't consistent across backends (yet!)

gforsyth avatar Jul 22 '24 17:07 gforsyth

Going to get started on this one. I think having a TableNotFound exception being raised across backends when there is no table is the most reasonable thing here.

@relmaazouz I'm not sure I follow your last comment.

ncclementi avatar Jul 25 '24 17:07 ncclementi

Great to see moving forwards!

A few thoughts:

  • Backwards compatibility: I've already got code which handles the different exceptions across the limited numbers of backends I support and I'm sure there will be other code out there too. This will break that. I don't think there's any way of preventing that from happening but I don't know what this project's compatibility policy is? (EDIT: Some sort of exception hierarchy might be possible, not tried it)
  • TableNotFound except should inherit from IbisError, which allows existing code relying on it to continue to work and I think is also still a sensible inheritance.
  • Where possible, I'd also raise from e, where e is the internal/backend error to persist the error message.

ssabdb avatar Jul 26 '24 08:07 ssabdb

Hey @ssabdb -- we try to follow SemVer quite strictly, so breaking changes will only happen in major releases.

There's probably a debate about whether unifying exceptions that we weren't catching before counts as a breaking change, but since we'll be adding a new exception type and changing that in a few places, I believe that will merit a breaking change tag, so it will come out in version 10.0

gforsyth avatar Jul 26 '24 13:07 gforsyth

There's probably a debate about whether unifying exceptions that we weren't catching before counts as a breaking change

Sure. IbisError is thrown by some of the backends so there's an argument it could be a breaking change anyway by that definition, but the outcome is the same.

ssabdb avatar Jul 26 '24 14:07 ssabdb