[RFE] Report which facts have been modified if using `fact_cache` param

[AlanCoding]

Using the combination of ansible-runner transmit, ansible-runner worker, and ansible-runner process, there's no really clean way to decipher which hosts have had facts updated. I have https://github.com/ansible/awx/pull/11641 up to make the AWX use work as intended, but it's not ideal

This is important in cases where gathering facts is expensive and lots of automation is being ran.

Request

This feature request is that ansible-runner delivers an artifact that gives a list of hosts which have had facts updated by the playbook run.

Implementation has to be based on file modification time, because that's how the jsonfile cache plugin works fundamentally. To give some pseudo code of how this can work:

• Before starting the job, the epoch time from time.time() is recorded
• After the job is finished, list all files in self.fact_cache (will need to fix for containerized jobs) and make a list of all filenames that have a modification time after the noted start time
• Return those values by writing it to a file in the artifacts folder, similar to status or rc

Additional Info

After discussing with @jbradberry, I agree that this implementation is a lot simpler than what AWX has evolved. However, I believe that doing this will involve other unplanned tech debt cleanup. For example, AWX sets the param "fact_cache_type"

https://github.com/ansible/awx/blob/397974fade453ccf41cb37b0a180f3e30a201ca3/awx/main/tasks/jobs.py#L707

...but it seems not not set "fact_cache", which means it's not truly operating through the ansible-runner fact mechanism. This would need to be fixed. Also, the RunnerConfig object from containerized jobs seems to (incorrectly) not set self.fact_cache.

[AlanCoding]

There are some other options to solve this problem, but I want to stress the point that it is solvable and it should be solved.

To clarify what problem this solves, I see two:

• the zipfile protocol will round down to the nearest 2 seconds (weird, I know!) so if an update to a file happened in under 2 seconds it might fail to record this (practically, unlikely)
• if time zones of control and execution nodes are off, the entire notion of transferring the modtime of the file when unzipping breaks down

For one alternative solution, you could backdate the original modtime of the facts files that are written to avoid any skew that sends that modtime into the past. If they files claim to have been written 1 week ago, then a timezone skew incorrectly marking it as 5 hours in the past won't affect the changed/not-changed determination.

As a supporting part of any of these solutions, we could create a reference file, in case you didn't trust that python's time.time() will give the same answer as the modification / creation times after writing a file. I don't know exactly why, but I understand the desire to only do apples-to-apples comparisons.

I believe there are more alternative solutions proposed. Any well-thought-out one should work. Ping @gamuniz

[AlanCoding]

I'm wondering if a simpler solution might be to have ansible-runner worker add a timestamp to the started callback data. That could use epoch time, I don't know if it matters. Then we could compare file modification times against that.

Right now, we compare against the file modtime on the control machine.

https://github.com/ansible/awx/blob/6d3c22e8010621dcd87845955b2042678fa0fadb/awx/main/tasks/facts.py#L57

I have a hesitation against comparing time.time() against file modification times, because it doesn't seem like it's apples-to-apples, but that's probably not an important point. Any form of timestamp from the execution node would be better.