package `pandas-gbq`

[frederikfab]

A lot of our ad hoc notebooks could be replaced with browser-based ones, if we could access google bigquery 👍

[martinRenou]

pandas-gbq is a noarch package on conda-forge meaning you can actually install it to be used in a browser-based notebook. Did you see issues with it?

[frederikfab]

I might be doing things wrong, but here are the steps i took:

docker run -it mambaorg/micromamba
# Inside docker container
micromamba create --name emscripten python=3.8 -c https://repo.mamba.pm/emscripten-forge -c https://repo.mamba.pm/conda-forge
micromamba activate emscripten
micromamba install pandas-gbq -c https://repo.mamba.pm/emscripten-forge -c https://repo.mamba.pm/conda-forge
python
# Inside python prompt
import pandas as pd
import pandas_gbq

And it throws an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/emscripten/lib/python3.8/site-packages/pandas_gbq/__init__.py", line 1, in <module>
    from .gbq import to_gbq, read_gbq  # noqa
  File "/opt/conda/envs/emscripten/lib/python3.8/site-packages/pandas_gbq/gbq.py", line 14, in <module>
    from pandas.compat import lzip, bytes_to_str
ImportError: cannot import name 'lzip' from 'pandas.compat' (/opt/conda/envs/emscripten/lib/python3.8/site-packages/pandas/compat/__init__.py)

Also happens for python version 3.10.

I never actually spun up the emscripten based jupyterlite environment. In a pyodide environment over at https://jupyterlite.readthedocs.io/en/latest/_static/lab/index.html i get the following output when trying to install this

import piplite
await piplite.install("pandas-gbq", keep_going=True)
(...)
ValueError: Can't find a pure Python 3 wheel for: 'pyarrow<10.0dev,>=3.0.0', 'pyarrow<10.0dev,>=3.0.0', 'pyarrow<10.0dev,>=3.0.0', 'grpcio<2.0dev,>=1.47.0', 'google-crc32c<2.0dev,>=1.0'
See: https://pyodide.org/en/stable/usage/faq.html#micropip-can-t-find-a-pure-python-wheel

These errors might all exist outside of this project, so my motivation for posting it here, is that i'm really excited by the idea of a conda emscripten arch target, as a way to expand the supported python packages, and this is the one blocker at the moment for us to adopt the browser-based notebooks at work.

[martinRenou]

I might be doing things wrong, but here are the steps i took:

It looks like you are not setting the target platform correctly for the micromamba environment. It needs to be an emscripten-32 platform

Note that you can easily build an emscripten env for usage in jupyterlite using the xeus-python kernel: https://xeus-python-kernel.readthedocs.io/en/latest/configuration.html No need to run the micromamba commands yourself, xeus-python will make sure to build the env and include it in the kernel so you can import your packages there.

---

I just tried with

micromamba install pandas-gbq -c https://repo.mamba.pm/emscripten-forge -c https://repo.mamba.pm/conda-forge

and it gives me:

                                           __
          __  ______ ___  ____ _____ ___  / /_  ____ _
         / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
      /_/

https://repo.mamba.pm/conda-forge/emscripten-32     93.0 B @  46.0 B/s 404 failed  2.0s
https://repo.mamba.pm/emscripten-forge/emscripte..  15.1kB @   7.5kB/s  2.0s
https://repo.mamba.pm/emscripten-forge/noarch      330.0 B @ 152.0 B/s  2.2s
https://repo.mamba.pm/conda-forge/noarch             9.7MB @   3.5MB/s  2.8s
Encountered problems while solving:
  - nothing provides protobuf >=3.0.0 needed by google-api-core-1.0.0-py_0

So it is not yet installable indeed. Let's see if we can get protobuf in.

[frederikfab]

Thanks for identifying that protobuf is what is needed here, and for pointing out how i can get a working jupyter lite environment up and running.

[jtpio]

So protobuf should be available via https://github.com/emscripten-forge/recipes/pull/454 now.

Maybe it means pandas-gbq can be packaged now.

[bastivnalfaro]

hola necesito ayuda con este error

ModuleNotFoundError: No module named 'google.cloud._helpers'

solo estoy tratando de importar la libreria pandas-gbq con buildozer para hacer mi apk pero me tira ese error

[DerThorsten]

hola necesito ayuda con este error

ModuleNotFoundError: No module named 'google.cloud._helpers'

solo estoy tratando de importar la libreria pandas-gbq con buildozer para hacer mi apk pero me tira ese error

An app build with bulldozer is usually for android. But here we build for wasm. This seems the appropriate repo https://github.com/kivy/buildozer

Una aplicación construida con Buildozer es generalmente para Android. Pero aquí estamos construyendo para WASM. Este parece ser el repositorio apropiado: https://github.com/kivy/buildozer

[martinRenou]

Maybe it means pandas-gbq can be packaged now

pandas-gbq is noarch on conda-forge, now that protobuf is packaged on emscripten-forge it should be already installable unless another dependency is required, but let's track those dependencies in specific package request issues if needed.

Closing as unrelevant anymore :)