Batuhan Taskaya

> Running it with on my local computer and not on a cluster. Are you using the same versions of everything (adlfs, fsspec) when running on the local computer?

I don't use `Dask`, but if you can provide a short reproducer with using the adlfs interace, I'd be happy to try and test it locally.

> In general, my understanding is that w/the move to using async instead of python thread pools (and aiobotocore) we should be able to start getting throughputs that are much...

I've been looking into this and noticed that there are currently two places where this needs to be handled (and from what I've understood, they are actually separate optimizations). The...

@Dandandan definitely! I've created #3596 for it.

> I think the only solution is to do two separate pip calls. @chamini2 but then we would end up overriding the `dbt-bigquery`'s dependency requirements, no? (or does `dbt-bigquery` just...

> Well, we don't really want to handle the exceptions, right? if it fails, it just fails and the model shows ERROR as status. I meant let the DBT handle...

One problem with this patch is that, we need to limit fetching from `__source__` only for the main frame, otherwise if the offsets match out it will show something like...

Thanks for the reminder @CAM-Gerlach, this was mainly an experiment before it ever got to be a real PR so yep we can close it.

@cktang88 that's true, it will run constantly until there is nothing to change. > I'm mostly concerned about the impact this has on performance over large codebases. This is something...