anaconda upload silently fails

[stas00]

Hi,

I'm experiencing weird issues with anaconda upload silently failing.

1. built a package:

conda skeleton pypi fastprogress
conda-build fastprogress

all was successful (this is just a pure python one-file module).

2. tried to upload:

anaconda upload fastprogress-0.1.5-py36_0.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "stason" as upload username
Processing 'fastprogress-0.1.5-py36_0.tar.bz2'
Detecting file type...
File type is "conda"
Extracting conda package attributes for upload
Creating package "fastprogress"

that's it. Nothing happens, no error message, nothing gets uploaded. (nothing appears under https://anaconda.org/stason/repo)

Please note that another team member was able to successfully upload this same package from his setup, but it doesn't work on mine:

conda-build 3.15.1
conda 4.5.11
anaconda Command line client (version 1.7.2)
Python 3.6.6 :: Anaconda custom (64-bit)
ubuntu 18.04

Thank you.

[goanpeca]

What version of anaconda client was your team member using?

[stas00]

His is Command line client (version 1.6.14)

But I can't see how that information would help me, if the client is not behaving properly. Surely it should have said that there is some problem, yet it does nothing. Does it make sense?

Thank you.

[stas00]

I did some more testing and was able to create (via conda skeleton pypi), build and upload other modules w/o any problems.

I also tested that this package fails with older anaconda client (1.6.5). So it must be something about the package.

You should be able to reproduce that on your side, since it was just the straightforward:

conda skeleton pypi fastprogress
conda-build fastprogress

[goanpeca]

@stas00 could you attach the package so I can take a look :-) ?

[stas00]

it won't let me upload this type of file, here it is on dropbox

And thank you for your help, @goanpeca

[stas00]

Were you able to get the package via dropbox, @goanpeca?

[goanpeca]

@stas00 I did, but I will take a look on monday!

[stas00]

some additional info:

• I'm able to upload this package under a different username -u fastai, a group, to which I'm allowed to upload to.
• I'm able to upload other packages under my own username -u stason

[mcg1969]

@goanpeca, @ericdill and I just discovered a silent failure wil occur in another instance as well: when using the --private flag on an organization (-u) that doesn't have private package privileges.

[mcg1969]

So my guess is there is some sort of permissions failure that goes undetected by the client.

[goanpeca]

@mcg1969 thanks for the tip. I will add some testing

[damianavila]

I have seen the same... I tried with anaconda-client 1.5.x and then I saw the issue, the token was expired... somehow the latest anaconda-client version eats that info when it fails, I think.

[goanpeca]

@damianavila yes, the clien has been eating a lot of error messages lately, the exception handling needs to be fixed.

[bryevdv]

This same issue (evidently) just botched a bokeh release. Tried restarting, did not help.

[hemenkapadia]

I am also facing a similar issue while uploading package to anaconda cloud

hemen  ~/Downloads/h2oai_clinet  anaconda -v --show-traceback upload -u h2oai h2oai_client-1.4.0-py36_d83ac6f_45.tar.bz2 
Using Anaconda API: https://api.anaconda.org
[DEBUG] Found login token: /home/hemen/.config/binstar/https%3A%2F%2Fapi.anaconda.org.token
Using "h2oai" as upload username
Processing 'h2oai_client-1.4.0-py36_d83ac6f_45.tar.bz2'
Detecting file type...
[DEBUG] Testing if conda package ..
[DEBUG] This is a conda package
File type is "conda"
Extracting conda package attributes for upload
Creating package "h2oai_client"
Creating release "1.4.0"
Uploading file "h2oai/h2oai_client/1.4.0/noarch/h2oai_client-1.4.0-py36_d83ac6f_45.tar.bz2"
 hemen  ~/Downloads/h2oai_clinet  1  conda verify h2oai_client-1.4.0-py36_d83ac6f_45.tar.bz2 
Verifying h2oai_client-1.4.0-py36_d83ac6f_45.tar.bz2...
 hemen  ~/Downloads/h2oai_clinet  echo $?
0
 hemen  ~/Downloads/h2oai_clinet  conda -v
usage: conda [-h] [-V] command ...
conda: error: the following arguments are required: command
 hemen  ~/Downloads/h2oai_clinet  2  conda -V
conda 4.5.11
 hemen  ~/Downloads/h2oai_clinet  conda build -V
conda-build 3.16.2
 hemen  ~/Downloads/h2oai_clinet  anaconda -V
anaconda Command line client (version 1.7.2)
 hemen  ~/Downloads/h2oai_clinet  

The page at (https://anaconda.org/h2oai) lists that package h2oai_client was created. However https://anaconda.org/h2oai/h2oai_client indicates no files exist in the package.

Any help on how to resolve this issue would be greatly appreciated.

[hemenkapadia]

I would consider this as a regression for anaconda clinet. Downgrading the client to 1.5.5 seems to resolve the issue. Now i see the package released to anaconda.org ...

https://anaconda.org/h2oai/h2oai_client/files

[bkreider]

I think i have narrowed this down. If you do a echo $? anaconda-client properly shows an error code of 1, but no output.

In upload.py:245 it needs to catch the exception. This snippet shows catching the exception, logging it and re-raising it:

        try:
            logger.info("About to upload")
            upload_info = aserver_api.upload(username, package_name, version, file_attrs['basename'], fd,
                                             binstar_package_type, args.description,
                                             dependencies=file_attrs.get('dependencies'), attrs=file_attrs['attrs'],
                                             channels=args.labels, callback=upload_print_callback(args))
            logger.info("after upload")
        except errors.Conflict:
            logger.info("Inside Except")
            upload_info = {}
            if args.mode != 'skip':
                logger.info('Distribution already exists. Please use the -i/--interactive or --force or --skip options '
                            'or `anaconda remove %s/%s/%s/%s', username, package_name, version, file_attrs['basename'])
                raise
            else:
                logger.info('Distribution already exists. Skipping upload.\n')
        except Exception as e:
                logger.info('Uncaught exception: %s' %  (str(e),))
                raise

The output looks like this:

Using Anaconda API: https://api.anaconda.org
Using "...." as upload username
Processing '/home/travis/miniconda/conda-bld/linux-64/......tar.bz2'
Detecting file type...
File type is "conda"
Extracting conda package attributes for upload
Creating package "......"
Creating release "1.5.1"
Uploading file "...."
About to upload
Uncaught exception: (u'Authentication token has expired', 401)

[bashtage]

Just got caught by @bkreider same issue -- token expired overnight and uploads were silently failing. Would really help to get a better error.

[mliu49]

Also ran into this issue. Travis build finished successfully but uploads were not showing up on anaconda.org.

Came across this and realized that the token had expired. Hopefully this can be fixed soon.

[vfdev-5]

I have a similar issue to upload a package to the organization where I have rights to upload, my token is valid. CMD error is 1 and no error message.

Thanks to @bkreider and his snippet, I could get the exact error with exceeded storage size.

[hmaarrfk]

Definitely annoying to have silent failures on permissions.

Using your trick @bkreider I was able to pinpoint that I needed write access to the API in addition to: "All operations on Conda Repositories"

Maybe this is just a private package issue?

[mcg1969]

@bkreider let's huddle on this very soon and work on getting a point release out there.

[ileasile]

Hello @mcg1969, I started to get silent failures a few days ago, without any changes in environment or package itself (tried building from the same branch). Upload looks like this:

C:\Projects\kotlin-jupyter>anaconda upload -u jetbrains-dev artifacts/kotlin-jupyter-kernel-0.8.2.53.dev3-py_0.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "jetbrains-dev" as upload username
Processing 'artifacts/kotlin-jupyter-kernel-0.8.2.53.dev3-py_0.tar.bz2'
Detecting file type...
File type is "conda"
Extracting conda package attributes for upload
Creating package "kotlin-jupyter-kernel"
Creating release "0.8.2.53.dev3"
Uploading file "jetbrains-dev/kotlin-jupyter-kernel/0.8.2.53.dev3/noarch/kotlin-jupyter-kernel-0.8.2.53.dev3-py_0.tar.bz2"

Process returns 1 exit code and gives no info about errors. This issue is reproducible in several different environments (locally and on CI in Docker container). My anaconda-client version is 1.7.2.

Could you please help me with investigating?

[ileasile]

I figured it out using technique mentioned by @bkreider :

Storage requirements exceeded (3221225472 bytes). Payment is required to add a file. Please go to https://anaconda.org/binstar.settings/billing to update your plan

[ndevenish]

I just had this issue also, silent failure; added the code from @bkreider and got:

Uncaught exception: module 'base64' has no attribute 'encodestring'

[piyueh]

I'm also having Uncaught exception: module 'base64' has no attribute 'encodestring'. The traceback is hidden by default because sys.excepthook is overwritten at line 90 in cli.py. To see the error traceback, in addition to @bkreider 's trick, I also had to comment out line 90 in cli.py. Here's the traceback I got:

Uncaught exception: module 'base64' has no attribute 'encodestring'
Traceback (most recent call last):
  File "<prefix>/bin/anaconda", line 11, in <module>
    sys.exit(main())
  File "<prefix>/lib/python3.9/site-packages/binstar_client/scripts/cli.py", line 159, in main
    binstar_main(command_module, args, exit,
  File "<prefix>/lib/python3.9/site-packages/binstar_client/scripts/cli.py", line 139, in binstar_main
    return args.main(args)
  File "<prefix>/lib/python3.9/site-packages/binstar_client/commands/upload.py", line 340, in main
    package_info = upload_package(
  File "<prefix>/lib/python3.9/site-packages/binstar_client/commands/upload.py", line 236, in upload_package
    upload_info = aserver_api.upload(username, package_name, version, file_attrs['basename'], fd,
  File "<prefix>/lib/python3.9/site-packages/binstar_client/__init__.py", line 530, in upload
    _hexmd5, b64md5, size = compute_hash(fd, size=size)
  File "<prefix>/lib/python3.9/site-packages/binstar_client/utils/__init__.py", line 52, in compute_hash
    b64encode = getattr(base64, 'encodebytes', base64.encodestring)
AttributeError: module 'base64' has no attribute 'encodestring'

Python version 3.9.5, installed through conda. anaconda-client's version is 1.7.1 py39h06a4308_0. Downgrade to Python 3.8, everything works fine then.

[Viech]

It's important to note that conda install anaconda-client will by default update Python, hence causing the hidden AttributeError: module 'base64' has no attribute 'encodestring' exception even if you are using, say, the conda/miniconda3 docker image that still ships Python 3.7.3 initially. You can avoid this by writing e.g. conda install python=3.8 anaconda-client.

If this does not help, see my comment in #564 on how to monkey-patch anaconda-client within a temporary conda environment using the patch proposed by @bkreider. This should give you an error message.