support token-based upload to PyPI

[brainwane]

Hi! PyPI's getting ready to add upload tokens to its API, so packagers can upload using tokens instead of username/password.

I thought you might want to test and verify that flit's upload works with this. Instructions for testing the pull request: https://github.com/pypa/warehouse/issues/994#issuecomment-512634222

It would be great if you could leave a comment on https://github.com/pypa/warehouse/pull/6084 telling us whether it works for you! This will help us avoid headaches when we roll out the feature on pypi.org.

[takluyver]

Thanks. I'm just about to go offline for a week, and then it will be another few days while I dig myself out from the emails that build up. But I'll try to get round to it.

[brainwane]

Please do enjoy your offline time! And if someone else would like to do the testing -- maybe @flying-sheep? -- that would, of course, be fine!

[brainwane]

This beta feature is now live on PyPI and Test PyPI: https://pypi.org/help/#apitoken

[Carreau]

Tried it; works great. Just set FLIT_USERNAME='@token' and FLIT_PASSWORD='<yourtoken>'.

One question we may want to adress is how we pick which token for which project if not in an env variable.

[takluyver]

Brainstorming how the UI could work:

If I'm trying to publish a release of austin and my ~/.pypirc says:

[pypi]
repository: https://upload.pypi.org/legacy/
username: takowl

Then flit could use the first it finds of:

# Project scoped token
keyring.get_password("https://upload.pypi.org/legacy/", "pypi-project-token:austin")

# User scoped token
keyring.get_password("https://upload.pypi.org/legacy/", "pypi-user-token:takowl")

If it doesn't find either of those, it would ideally connect as my user, prompting me for any necessary credentials, and then obtain and save a project-scoped token to use for uploads. If that's not possible, it might fall back to uploading with username & password as it currently does.

Does that scheme fit in with how you'd like to use tokens? Of course, it could all be overridden by env variables for CI - I'm just thinking about the upload-from-local case.

[brainwane]

cc @di and @ewdurbin and @dstufft to answer this question. @woodruffw as an FYI.

[Carreau]

The idea would work with me I had something similar in mind wihtout exact details; I'd just like the inerface to be (more or less) standardized across tools that can publish packages. That give us a chance to – at some point – have a tools to list and manipulate tokens.

Thus I could not trust flit ; and do somehting like pip add-token austin <token>, and flit would be able to pick it up.

[takluyver]

It would certainly be good if we could ensure different tools can use the same credentials. I think flit and twine can currently use the same usernames & passwords in keyring, since there's one obvious way to store that. It might take a bit more coordination for tokens. Or maybe flit can just use twine.

[brainwane]

Hi - this feature is now out of beta on PyPI, so I hope you can officially support it!

[brainwane]

Changed the title to reflect that this is now a feature and documentation request.

[kown7]

Tried it; worg great. Just set FLIT_USERNAME='@token' and FLIT_PASSWORD='<yourtoken>'.

I had to use FLIT_USERNAME=__token__

[woodruffw]

I had to use FLIT_USERNAME=__token__

Yep, this makes sense: the @token name was removed before the beta ended.

[kown7]

Is there some documentation planned for this?

[ewdurbin]

This is currently documented at https://packaging.python.org/specifications/pypirc/?highlight=token#using-a-pypi-token and https://pypi.org/help/#apitoken

[chrisjsewell]

FYI, I do this in a Github action (triggered by a tagged release), so you can then keep the token in a repository secret:

  publish:
    name: Publish to PyPi
    needs: [pre-commit, tests]
    if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
    runs-on: ubuntu-latest
    steps:
    - name: Checkout source
      uses: actions/checkout@v2
    - name: Set up Python 3.7
      uses: actions/setup-python@v1
      with:
        python-version: 3.7
    - name: install flit
      run: |
        pip install flit~=3.0
    - name: Build and publish
      run: |
        flit publish
      env:
        FLIT_USERNAME: __token__
        FLIT_PASSWORD: ${{ secrets.PYPI_KEY }}

see for example: https://github.com/executablebooks/mdformat-tables/blob/a83ba37580c8f6c1827bce59a285ea62f47f56d5/.github/workflows/tests.yml#L74

[Carreau]

For crosslink I'm going to add https://github.com/pypa/twine/issues/496 and https://github.com/pypa/twine/issues/565 as relevant.

In particular I'm when creating a token warehouse display this example for twine:

[distutils]
  index-servers =
    pypi
    PROJECT_NAME

[pypi]
  username = __token__
  password = # either a user-scoped token or a project-scoped token you want to set as the default

[PROJECT_NAME]
  repository = https://upload.pypi.org/legacy/
  username = __token__
  password = # a project token 

[Mariatta]

I've added some instructions on how to publish a package using PyPI token. https://github.com/pypa/flit/pull/602