grayskull
grayskull copied to clipboard
conda
Allow passing a package mapping for pypi->conda-forge packages
We should have some way of having a mapping where pypi package names (such as docker vs docker-py) can be mapped from pypi names to conda names.
We do have it on grayskull :)
Please see
https://github.com/marcelotrevisani/grayskull/blob/master/grayskull/pypi/config.yaml
So we will need to just add a new entry to that file, something like
docker:
import_name: docker
conda_forge: docker-py
Do you know other differences of project names between pypi and conda-forge? I just have a few in that file
I don't know what you mean, sorry Can you explain a bit more please?
going to take a shot at generating this automatically from our metadata in https://github.com/regro/cf-graph-countyfair/tree/master/node_attrs
So looking at the conda-forge metadata i can generate a list of 174 packagest that have a non-case mismatch between conda-forge and pypi name.
we can probably generate this automatically as part of the conda-forge bot and publish it to github somewhere?
Automatically found mismatches
Boruta:
conda_forge: boruta_py
import_name: boruta
Flask-Cors:
conda_forge: flask_cors
import_name: flask_cors
Flask-JSON:
conda_forge: flask_json
import_name: flask_json
SoundFile:
conda_forge: pysoundfile
import_name: soundfile
aiohttp_cors:
conda_forge: aiohttp-cors
import_name: aiohttp_cors
backtrace:
conda_forge: python-backtrace
import_name: backtrace
bash_completion:
conda_forge: py-bash-completion
import_name: bash_completion
captest:
conda_forge: pvcaptest
import_name: captest
cdo:
conda_forge: python-cdo
import_name: cdo
click_config_file:
conda_forge: click-config-file
import_name: click_config_file
cockroachdb:
conda_forge: cockroachdb-python
import_name: cockroachdb
conda_lock:
conda_forge: conda-lock
import_name: conda_lock
conda_mirror:
conda_forge: conda-mirror
import_name: conda_mirror
coreapi:
conda_forge: python-coreapi
import_name: coreapi
coreschema:
conda_forge: python-coreschema
import_name: coreschema
cryptography_vectors:
conda_forge: cryptography-vectors
import_name: cryptography_vectors
cufflinks:
conda_forge: python-cufflinks
import_name: cufflinks
dask:
conda_forge: dask-core
import_name: dask
datadotworld:
conda_forge: datadotworld-py
import_name: datadotworld
datalad_container:
conda_forge: datalad-container
import_name: datalad_container
delegator.py:
conda_forge: delegator
import_name: delegator
dials_data:
conda_forge: dials-data
import_name: dials_data
dye_score:
conda_forge: dye-score
import_name: dye_score
empyrical_dist:
conda_forge: empyrical-dist
import_name: empyrical_dist
esprima:
conda_forge: esprima-python
import_name: esprima
et_xmlfile:
conda_forge: et-xmlfile
import_name: et_xmlfile
evalml:
conda_forge: evaml-core
import_name: evalml
extract-msg:
conda_forge: msg-extractor
import_name: extract_msg
flake8_nb:
conda_forge: flake8-nb
import_name: flake8_nb
flatbuffers:
conda_forge: python-flatbuffers
import_name: flatbuffers
flex:
conda_forge: flex-swagger
import_name: flex
gnupg:
conda_forge: gnupg-py
import_name: gnupg
google:
conda_forge: googlesearch
import_name: googlesearch
graphviz:
conda_forge: python-graphviz
import_name: graphviz
h2o:
conda_forge: h2o-py
import_name: h2o
hdfs:
conda_forge: python-hdfs
import_name: hdfs
iam_units:
conda_forge: iam-units
import_name: iam_units
ib_insync:
conda_forge: ib-insync
import_name: ib_insync
intake-astro:
conda_forge: intake-accumulo
import_name: intake_astro
intake_geopandas:
conda_forge: intake-geopandas
import_name: intake_geopandas
ioos_tools:
conda_forge: ioos-tools
import_name: ioos_tools
jupyter_jaeger:
conda_forge: jupyter-jaeger
import_name: jupyter_jaeger
jupyterlab_git:
conda_forge: jupyterlab-git
import_name: jupyterlab_git
jupyterlab_latex:
conda_forge: jupyterlab-latex
import_name: jupyterlab_latex
libaio:
conda_forge: python-libaio
import_name: libaio
md_toc:
conda_forge: md-toc
import_name: md_toc
message_ix:
conda_forge: message-ix
import_name: message_ix
mss:
conda_forge: python-mss
import_name: mss
nbconvert_utils:
conda_forge: nbconvert-utils
import_name: nbconvert_utils
ndarray_listener:
conda_forge: ndarray-listener
import_name: ndarray_listener
neo:
conda_forge: python-neo
import_name: neo
nest_asyncio:
conda_forge: nest-asyncio
import_name: nest_asyncio
node-semver:
conda_forge: python-node-semver
import_name: semver
nvidia-ml-py3:
conda_forge: nvidia-ml
import_name: pynvml
pandas_flavor:
conda_forge: pandas-flavor
import_name: pandas_flavor
paragraph:
conda_forge: python-paragraph
import_name: paragraph
prince:
conda_forge: prince-factor-analysis
import_name: prince
pvlib:
conda_forge: pvlib-python
import_name: pvlib
pyct:
conda_forge: pyct-core
import_name: pyct
pytest_check_links:
conda_forge: pytest-check-links
import_name: pytest_check_links
python-datamatrix:
conda_forge: datamatrix
import_name: datamatrix
python-fileinspector:
conda_forge: fileinspector
import_name: fileinspector
python-pseudorandom:
conda_forge: pseudorandom
import_name: pseudorandom
python-pygaze:
conda_forge: pygaze
import_name: pygaze
python-qdatamatrix:
conda_forge: qdatamatrix
import_name: qdatamatrix
python-qnotifications:
conda_forge: qnotifications
import_name: QNotifications
quantum-grove:
conda_forge: grove
import_name: grove
radio_beam:
conda_forge: radio-beam
import_name: radio_beam
rocketpyalpha:
conda_forge: rocketpy
import_name: rocketpy
sage_flatsurf:
conda_forge: sage-flatsurf
import_name: flatsurf
sagemaker_inference:
conda_forge: sagemaker-inference-toolkit
import_name: sagemaker_inference
sagemaker_mxnet_training:
conda_forge: sagemaker_mxnet_container
import_name: sagemaker_mxnet_container
sounddevice:
conda_forge: python-sounddevice
import_name: sounddevice
spherical-functions:
conda_forge: spherical_functions
import_name: spherical_functions
stjudecloud-oliver:
conda_forge: oliver
import_name: oliver
termstyle:
conda_forge: python-termstyle
import_name: termstyle
torch_geometric:
conda_forge: pytorch_geometric
import_name: torch_geometric
useDAVE:
conda_forge: dave
import_name: numpy
vsts:
conda_forge: vsts-python-api
import_name: vsts
it would be good if this listing was independent of grayskull itself, so we can generate it dynamically
we can probably generate this automatically as part of the conda-forge bot and publish it to github somewhere?
That would be nice indeed
I have a webservice in my personal vps which I use it sometimes to just generate the recipe do a project on pypi using grayskull for me It is possible to create an api to retrieve that list Actually there is an api there already, but it just retrieve the recipe, I can extended or create a new service just for this part
Okay we have started producing this mapping as part of the bot. It should update roughly hourly
https://github.com/regro/cf-graph-countyfair/blob/master/mappings/pypi/grayskull_pypi_mapping.yaml
Hi, just to check; what's currently the correct place to add such quirks? Is it still https://github.com/conda-incubator/grayskull/blob/master/grayskull/pypi/config.yaml?
Reason is that I would like to add fastjsonschema (fastjsonschema vs python-fastjsonschema) and flair (flair vs python-flair).
Hi, just to check; what's currently the correct place to add such quirks? Is it still https://github.com/conda-incubator/grayskull/blob/master/grayskull/pypi/config.yaml?
Reason is that I would like to add fastjsonschema (
fastjsonschemavspython-fastjsonschema) and flair (flairvspython-flair).
Hi @BastianZim , Yes, that is correct. You will need to add it there, just follow the previous entries and it should be fine. :)
Hmm, I worked around this somewhat manually using conda-souschef (https://github.com/marcelotrevisani/souschef/issues/32). Not sure I understand the idea behind using config.yaml as described here. Is the idea to do a local installation of grayskull and edit that file? Also https://github.com/conda- incubator/grayskull/blob/master/grayskull/pypi/config.yaml redirects to a dead link: https://github.com/conda-incubator/grayskull/blob/main/grayskull/pypi/config.yaml.
Sometimes, the names in conda-forge are different than in PyPI. What this config.yaml does is it saves all of the differences and then when you run Grayskull on a package, it will give out the correct dependency.
For example, let's assume you have package x which has a dependency called y but on conda-forge the dependency is called py-y. When you now run Grayskull on x the dependency list will include py-y and not y.
@BastianZim where should config.yaml be placed? Should I make a folder structure in the project: grayskull/pypi/config.yaml?
Didn't see any files on grayskull called config.yaml: https://github.com/conda-incubator/grayskull/search?q=config.yaml
More just wondering how/if I should be using config.yaml.
As I mentioned, my workaround has been to manually replace those Anaconda/PyPI naming differences using conda-souschef, but didn't realize there might already be some functionality for that.
The file already exists, it was just moved to here: https://github.com/conda-incubator/grayskull/blob/main/grayskull/strategy/config.yaml
And yeah, GitHub search isn't always the best...
You can use souschef but if you add the respective package to the "official" yaml, others will be able to use it as well.
@BastianZim this is great! Thanks for letting me know, and good to know about GitHub search. Didn't realize that - will probably stick with VS Code Ctrl+Shift+F when I want to be sure. Will open a PR for kaleido --> python-kaleido