feat: Consistent tablenotfound exception across backends
Is your feature request related to a problem?
Different backends expose tables not being found in different ways.
For example,
connection = ibis.connect("pandas://")
connection.table("missing-table")
KeyError: 'missing-table'
connection = ibis.connect("bigquery://....")
connection.table("missing-table")
google.api_core.exceptions.NotFound: '...'
What is the motivation behind your request?
What I'd love to be able to do is to catch a consistent error in the handling code which is agnostic of the underlying backend, e.g.
def my_function(connection):
try:
connection.table("missing-table")
except IbisTableNotFound:
<handle cleanly>
Describe the solution you'd like
.
What version of ibis are you running?
9.0.0
What backend(s) are you using, if any?
DuckDB, Bigquery
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
+1, definitely a good idea!
This was added on the 9.1.0 version:
It raises IbisError on postgres, but:
- IbisError seems like a generic exception, not an exception which is explicit about TableNotFound. A table not existing is sufficiently generic and common across backends that (in my opinion) it should be part of the API.
- This definitely isn't the case with the bigquery backend even in 10.x, when released. I don't think that's an error with Bigquery, I think that's inconsistency across backends on how to catch missing tables.
yeah, this isn't consistent across backends (yet!)
Going to get started on this one. I think having a TableNotFound exception being raised across backends when there is no table is the most reasonable thing here.
@relmaazouz I'm not sure I follow your last comment.
Great to see moving forwards!
A few thoughts:
- Backwards compatibility: I've already got code which handles the different exceptions across the limited numbers of backends I support and I'm sure there will be other code out there too. This will break that. I don't think there's any way of preventing that from happening but I don't know what this project's compatibility policy is? (EDIT: Some sort of exception hierarchy might be possible, not tried it)
- TableNotFound except should inherit from IbisError, which allows existing code relying on it to continue to work and I think is also still a sensible inheritance.
- Where possible, I'd also
raise from e, where e is the internal/backend error to persist the error message.
Hey @ssabdb -- we try to follow SemVer quite strictly, so breaking changes will only happen in major releases.
There's probably a debate about whether unifying exceptions that we weren't catching before counts as a breaking change, but since we'll be adding a new exception type and changing that in a few places, I believe that will merit a breaking change tag, so it will come out in version 10.0
There's probably a debate about whether unifying exceptions that we weren't catching before counts as a breaking change
Sure. IbisError is thrown by some of the backends so there's an argument it could be a breaking change anyway by that definition, but the outcome is the same.