Enhancement: "--skip-existing" for running "flit publish" (e.g., within CI)

[cjrh]

Apologies if this feature already exists. I didn't find it.

I'd like to run flit publish automatically at the end of a CI build. If the current version already exists on pypi, the publish command will produce an error, but if there was a --skip-existing flag, then such errors could be suppressed. For example, for setuptools-based projects I use a basic appveyor.yml template that looks something like this (the pypi deploy part is at the bottom):

environment:

  username: cjrh
  password:
    secure: XXX

  matrix:

    - PYTHON: "C:\\Python27"
    - PYTHON: "C:\\Python27-x64"
    - PYTHON: "C:\\Python35"
    - PYTHON: "C:\\Python35-x64"
    - PYTHON: "C:\\Python36"
    - PYTHON: "C:\\Python36-x64"

install:
  # We need wheel installed to build wheels
  - "%PYTHON%\\python.exe -m pip install -U pip setuptools wheel"
  - "%PYTHON%\\python.exe -m pip install requirements-test.txt"

build: off

test_script:
  - "%PYTHON%\\python.exe setup.py test"

after_test:
  - "%PYTHON%\\python.exe setup.py bdist_wheel"

artifacts:
  - path: dist\*

deploy_script:
  - "echo [pypi] > %USERPROFILE%\\.pypirc"
  - "echo username: %username% >> %USERPROFILE%\\.pypirc"
  - "echo password: %password% >> %USERPROFILE%\\.pypirc"
  - "%PYTHON%\\python.exe setup.py bdist_wheel"
  - "%PYTHON%\\python.exe -m pip install twine"
  - "%PYTHON%\\python.exe -m twine upload --skip-existing dist/*"

So basically: --skip-existing will only upload the build artifacts if they're new.

[Carreau]

Thanks @cjrh ;

I tend to think this will be more useful as an utility "Does this version already exist" which would allow to conditionally use flit. Now the hard part is to get the information about the version numbers and filename from flit, maybe a dry-run option ? With a --json to make it easy to parse ?

You might also be able to just build with flit, and publish with twine; in which case you can already make use of all the flags present in twine.

[cjrh]

Yup, that's a clever idea: just deploy with twine. That will work fine for me. Feel free to close this.

[Carreau]

Leaving open, to do at least a documentation update/FAQ. If it's often requested we could also add the option, but let's get some data first :-)

[takluyver]

Another approach is to deploy only when the build is on a git tag (Travis makes this straightforward, not sure about Appveyor).

I'll agree with @Carreau - it may make sense to add it, but as there's an easy workaround (using twine), there's no rush.

Maybe flit publish will stay for simple use cases, and point people to twine for more options. Maybe flit publish will become a wrapper around twine, rather than reimplementing the HTTP communications. There are advantages to having it all in one place, though.

[cjrh]

And yet another option might be that flit simply uses twine as a library, maybe passes unknown (to flit) cmdline args onto twine.cli.dispatch(). [edit: actually, this is not a new idea -> your wrapper comment]

But anyway, the "right thing" is probably to not do anything until enough users want it :)

[kairichard]

I would want such a feature inside a CI/CD secenario. Currently we solve that by using flit publish || exit 0 which suboptimal.