Topics on testing

[aaronpowell]

Recently I've been spending time digging into creating tests for an Aspire application and looking at the way we would tackle some complex testing scenarios of a distributed application.

While we have a doc on testing I have been running into a variety of questions that I've had to answer, and I think they would make for good additions to our current content.

Here's the question I've been trying to answer:

• [ ] Do I create the AppHost once, or for each test? (what is the guidance for a test pattern)
• [ ] How do I provide config/user-secrets to app host? (it's not setting them on the test project, but the AppHost project, which isn't obvious)
• [ ] How do I do/work with the logs to diagnose test failures? (we can't access the dashboard, so what's the alternative)
• [ ] How do I access non-HTTP resources? (there's an API)
• [ ] How do I best use the ResourceNotificationService?
• [ ] How do I integrate with Playwright for UI automation tests? (true that this isn't necessarily Aspire related, but Aspire means you can get the whole app stood up, but also, getting the HTTP endpoint of services it a bit clunky)
• [ ] What considerations do I need for running Aspire tests on GitHub Actions? Will running a bunch of containers be a problem? Should I use Using containerized services - GitHub Docs rather than running my own containers? What if I need Azure/AWS resources?
  • [ ] There could be a similar topic on Azure Pipelines

[aaronpowell]

@IEvangelist do you want me to turn this into a task list? I'm going to get started on producing some of these so am wondering if you want a single roll-up issue or more fine grain tracking.

[IEvangelist]

@IEvangelist do you want me to turn this into a task list? I'm going to get started on producing some of these so am wondering if you want a single roll-up issue or more fine grain tracking.

Task list please, that way we can create individual issues/PRs and I could potentially help.

[aaronpowell]

Done.

[aaronpowell]

I'm starting to tackle this content and looking at how we can have it structured appropriately. Given there's currently only a single top-level page for Testing, the first step would be looking to transition into a list structure. Here's my thought:

Testing
  - Tutorial - Writing your first test (the current page on Testing)
  - Managing the App Host (covers how to share the App Host between tests and managing user secrets)
  - Accessing resources (covers waiting for resources with IResourceNotificationService and accessing non-HTTP resources)
  - Diagnostics and logging (Not super sure on if this would be needed, but it's mostly how to get the structured log output)
  - Integrating with CI (GitHub Actions and Azure Pipelines)

Does this structure look ok? My priority would be the first three pages as they cover the most important parts of testing with Aspire while the others are more "nice to have" and will be more relevant as everything matures. Another future area to explore would be custom integration testing, which could live under the Testing or Custom integration area.

[aaronpowell]

@IEvangelist can we re-open this (I shouldn't have tagged it as fixed with #1874) as there's a few other articles I think we can do under the topic of testing