Add support for testing pebble services interaction

[tonyandrewmeyer]

There is currently no way in ops-testing to test a charm interaction with a pebble service, like restarting it for example: self._container.restart(self._service_name).

The only way to accomplish this is to mock ops.model.Container.restart.

It would be nice to have a better way to do this, maybe as an automatic mock in the output state?

...

state_out = self.ctx.run(container.pebble_ready_event, state_in)

assert state_out.containers[0].services["ausf"].restarted

Moved from https://github.com/canonical/ops-scenario/issues/34

[benhoyt]

Let's have someone spend 1-2h before end of Sept to see if we can determine from charm unit tests whether this is "missing" or how they're doing it now. @tonyandrewmeyer to do this.

@dimaqq Also to provide his higher-level idea.

[sinapah]

I also think this would be very helpful as it may be valuable to know whether a restart has happened. A specific use case would be when you are asserting that an event results in a restart of a particular service. Somewhat related would be to be able to get an equivalent of pebble changes so you can see all the replans and restarts that took place.

[tonyandrewmeyer]

Thanks @sinapah ! I think that's a fairly indicative example of where things are now. You can test to a certain level, if you're willing to assume things like "replan means services go to active" but you can't test more complex cases.

From what I've found, mostly charms just don't test this right now (or test things via integration tests instead).

We don't have space for this in 26.04 at the moment, but I think it's worth keeping this open for when we do have time.

[dimaqq]

From memory, charmers come to this wanting to test code like:

• on specific event, we know what things have changed
• new plan is generated
• service is explicitly restarted
• let's assert that it was restarted

However, a reconciler charm would behave more like:

• on any event
• plan is generated
• plan is applied
• container-side pebble decides if service is actually restarted

The earlier is more amenable to testing, but is it the charm style that we want to encourage?

The latter (I think) is already testable via "assert applied_plan == expected_plan". Testing that deeper would require a custom workload mock, which would need API design:

• given env var X in the plan, health checks are expected to pass
• in some corner case, health checks are expected to fail
• if service A (of N available) gets restarted as part of replan, expect a custom notice later