fedora-coreos-pinger icon indicating copy to clipboard operation
fedora-coreos-pinger copied to clipboard

*: Data collection from FCOS machines

Open zonggen opened this issue 6 years ago • 10 comments

Source of truth for each part of data collected:

  • Minimal:
    • platform: read_file /proc/cmdline
    • current_os_version: read_output rpm-ostree status --json
    • original_os_version: read_file /.coreos-aleph-version.json
    • instance_type: read_file /run/metadata/afterburn
  • Full:
    • hardware:
      • disk: read_output lsblk --fs --json
      • cpu: read_output lscpu --json
      • memory: read_output lsmem --json
    • network: read_output nmcli device show
    • container_runtime:
      • podman:
        • check if running by pgrep podman
        • count running container by pgrep conmon | wc -l
      • docker:
        • check if running by pgrep dockerd
        • count running container by pgrep containerd-shim | wc -l
      • cri-o:
        • check if running by pgrep crio
        • count running container by pgrep crictl | wc -l
      • system-nspawn:
        • check if running and counting containers by pgrep systemd-nspawn

Signed-off-by: Allen Bai, [email protected]

zonggen avatar Sep 18 '19 20:09 zonggen

Currently in main.rs the collected ignition platform id will be printed to the console, which will be updated later to be bundled with other data and sent to the remote endpoint.

zonggen avatar Sep 18 '19 20:09 zonggen

Also, this PR will be updated to resolve https://github.com/coreos/fedora-coreos-pinger/issues/30#issue-494138139

zonggen avatar Sep 18 '19 20:09 zonggen

UPDATE:

  • Added the module os-release.rs under identity to extract OS version from /etc/os-release

Note: Most portion of code is the same as platform.rs so in the future, towards the end of this PR should de-duplicate this part of code into a new utils helper module.

zonggen avatar Sep 19 '19 21:09 zonggen

STATUS: Currently Travis CI is failing since it is using Rust docker image which is based on Debian and it does not ship with lsmem, and lscpu does not contain an --json option. To test with more production-like environment, i.e. FCOS, it might be better to switch to UBI (https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image). Will be updating with changes to see if it works.

EDIT:

  • Updated .travis.tml to use build a container using Dockerfile and test the code inside the UBI container

zonggen avatar Oct 15 '19 21:10 zonggen

STATUS:

  • Since collecting service is meant to running in the background and users should not expect visible error information, the program should not crash/return error when one of the commands returns non-zero and instead should suppress the error and collect whatever is available at the moment.
  • Could use a log file to dump the error message

zonggen avatar Oct 21 '19 16:10 zonggen

STATUS:

  • Client side data collection is done. though the send_data() function is not doing anything other than printing the collected data in Json format. This needs to be updated after we have an actual endpoint.
  • Persistent state so that pinger could send data periodically with some fixed interval will be added through another PR to reduce the size of this PR

Overall, ready for review!

zonggen avatar Oct 22 '19 16:10 zonggen

STATUS:

  • Built on top of https://github.com/coreos/fedora-coreos-pinger/pull/35, in the main function, to create a Daemon process, one process is forked and parent process is immediately terminated. Two threads are then spawned by child process to track the time for daily and monthly reports.
  • The two threads are put into sleep to avoid busy waiting, and only checks whether it needs to send new report every 12 hours / 15 days for daily and monthly reports correspondingly.

zonggen avatar Oct 24 '19 20:10 zonggen

cc @lucab @bgilbert

zonggen avatar Oct 28 '19 13:10 zonggen

Is this something we want to continue pursuing?

ashcrow avatar Mar 18 '21 13:03 ashcrow

Maybe not; xref https://github.com/coreos/fedora-coreos-tracker/issues/770.

bgilbert avatar Mar 18 '21 19:03 bgilbert