setup testing workflow for Quickstart submissions

[jpvajda]

We need a quick and easy way to test new quickstarts and ensure they work in the product.

When a new Quick start is submitted, or an existing one is changed how do we validate the Quickstart actually does what it's supposed to do in the product?

Considerations

• can we use the import.sh tool defined here?
• can we setup a test account where we can safely install these quick starts for validation?
• how much of this is worth automating?

What would we want to test?

• Can we test the install plan they are using to ensure if they wanted to use a nerdlet or targeted install, we'd want to verify that the parameters they're using will actually result in a functional flow?
• Can we test to ensure a link reference actually routes and doesn't 404?
• Can we test Dashboard NRQL and Alert syntax?

[jpvajda]

@rudouglas you had some good ideas for this need, so I made a 🎫

[rudouglas]

Chatted to @jbeveland27 on this:

I don't know of any plans we have for improving it currently. It was really meant to be a quick way to test packs for the futurehack. I've talked previously about building a packs command into the newrelic-cli but it's never been prioritized. I'd like us to do something similar to what we do with the open-install-library, where a PR check would run the API calls to NR on a test account and then verify that things were created as expected. That's a bit of work to get going though and I'm not sure who makes the call as to assigning that as worthwhile work

This may be work we should consider doing as it would automate even more of the PR process.

Virtuoso also have an internal ticket to add Quickstarts API support to the cli which could also be considered VIRTUOSO-1270

[jpvajda]

Cool good info. I feel this a good post GA follow up. So we can address it later.

[github-actions[bot]]

Old issues will be closed after 105 days of inactivity. This issue has been quiet for 90 days and is being marked as stale. Reply here to keep this issue open.

[github-actions[bot]]

Old issues will be closed after 105 days of inactivity. This issue has been quiet for 90 days and is being marked as stale. Reply here to keep this issue open.

[github-actions[bot]]

Old issues will be closed after 105 days of inactivity. This issue has been quiet for 90 days and is being marked as stale. Reply here to keep this issue open.

[github-actions[bot]]

This issue is being closed due to inactivity. Is this a mistake? Please re-open this issue or create a new one.