iterative
Feature Request: Private Links for `cml publish`
I'm still learning cml, I noticed when I run:
cml publish dvc_plots/index.html
It creates a url at https://asset.cml.dev/ with a token. I'm using a private repo, so it was a little alarming to see that my data is being uploaded to a publicly available site, previously I thought it must be using some gitlab/github internals to upload the image to the mr/pr that I didn't yet understand.
In my case I'm using gcs as a dvc remote, so I think my workaround will be to instead upload the results to gcs then only people logged into a google account with access to the gcp project will have access.
It would be great if it was made clearer in the documentation that cml publish is uploading to cml.dev and if my use case with a private dvc remote is pretty typical it would be nice to provide a flag to cml publish that let's me instead upload to my private infrastructure.
if you are using GitLab there is a --native flag you can use, additionally there is a --url component to do exactly that. It is lacking some documentation as we haven't really put together a guide for its setup.
example
Here is an internal test using this feature which you can use/snoop around with:
jobs:
setup:
outputs:
url: ${{ steps.deploy.outputs.url }}
runs-on: ubuntu-latest
environment: test
steps:
- uses: actions/checkout@v3
with:
repository: 0x2b3bfa0/minroud
- name: Install Deps
run: npm install --global serverless
- uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: arn:aws:iam::342840881361:role/SandboxUser
aws-region: us-east-1
- name: Deploy
id: deploy
run: |
npx serverless deploy
url=$(npx serverless info | awk 'match($0, "endpoint: "){ print substr($0, RSTART + 10)}')
echo "$url"
echo "::set-output name=url::$url"
- uses: actions/upload-artifact@v3
with:
name: state
path: .serverless
test:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Test a CML Branch
if: ${{ env.TEST_CML_BRANCH }}
run: |
npm install -g canvas@2 vega@5 vega-cli@5 vega-lite@5 github:iterative/cml#${{ env.TEST_CML_BRANCH }}
- name: Standard CML setup
if: ${{ !env.TEST_CML_BRANCH }}
uses: iterative/setup-cml@v1
with:
version: ${{ env.TEST_CML_VERSION || 'latest' }}
- name: Test
env:
CML_PUBLISH_URL: ${{ needs.setup.outputs.url }}
run: |
echo "hello" >> file.md
url=$(cml publish file.md)
echo "$url"
test "hello" == $(curl --silent $url)
clean-up:
runs-on: ubuntu-latest
if: ${{ always() }}
needs: test
environment: test
steps:
- uses: actions/checkout@v3
with:
repository: 0x2b3bfa0/minroud
- name: Install Deps
run: npm install --global serverless
- uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: arn:aws:iam::342840881361:role/SandboxUser
aws-region: us-east-1
- uses: actions/download-artifact@v3
with:
name: state
path: .serverless
- name: Clean up
run: |
bucket_name=$(aws s3 ls | awk 'match($0, /minroud/){ print $3}')
aws s3 rm --recursive "s3://$bucket_name"
npx serverless remove
https://github.com/iterative/cml/blob/e87a23b11e12d72a0a8c7488348db930b4dcc121/bin/cml/publish.js#L35-L39
- [x] Amazon Web Services: https://github.com/iterative/minroud-s3
- [ ] Google Cloud
- [ ] Microsoft Azure
/?
@0x2b3bfa0 we have a (doubly hidden) publish --url now, but we're missing a public API.
i.e. we still need cml comment {create,update} --publish-url
fixed by https://github.com/iterative/cml.dev/pull/324? Or was there some other reason you re-opened @0x2b3bfa0? If so please do re-open (again)
The reason why I reopened this issue is because Minroud doesn't produce “private” links in the sense that knowing the URL is enough.