FreeBSD Support

[redanthrax]

Updated support for FreeBSD using dbolcsfoldi's work from #40.

Sample host data

{
  "host.cpu": {
    "idle": 84740000000,
    "irq": 3400000000,
    "system": 15410000000,
    "user": 43490000000
  },
  "host.info": {
    "architecture": "amd64",
    "boot_time": "2024-04-11T07:17:11.943687Z",
    "id": "535510ba-782f-4398-a566-76cef038cad1",
    "ip": [
      "fe80::5054:ff:fe49:8a64/64",
      "192.168.122.66/24",
      "::1/128",
      "fe80::1/64",
      "127.0.0.1/8"
    ],
    "kernel_version": "14.0-RELEASE",
    "mac": [
      "52:54:00:49:8a:64"
    ],
    "name": "freebsd",
    "native_architecture": "",
    "os": {
      "build": "RELEASE",
      "family": "freebsd",
      "major": 14,
      "minor": 0,
      "name": "FreeBSD",
      "patch": 0,
      "platform": "freebsd",
      "type": "freebsd",
      "version": "14.0-RELEASE"
    },
    "timezone": "UTC",
    "timezone_offset_sec": 0
  },
  "host.memory": {
    "available_bytes": 6008164864,
    "free_bytes": 5357379584,
    "raw": {
      "active_bytes": 29511680,
      "buffer_bytes": 608727552,
      "cache_bytes": 0,
      "free_bytes": 5357379584,
      "inactive_bytes": 42057728,
      "wired_bytes": 774049792
    },
    "total_bytes": 6402035712,
    "used_bytes": 803561472,
    "virtual_free_bytes": 1073610752,
    "virtual_total_bytes": 1073610752,
    "virtual_used_bytes": 0
  }
}

[cla-checker-service[bot]]

💚 CLA has been signed

[redanthrax]

@dbolcsfoldi Can you sign the agreement to get your commits in?

[elasticmachine]

:grey_exclamation: Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-08-25T20:31:24.259+0000

• Duration: 2 min 58 sec

Steps errors

Expand to view the steps failures

Load a resource file from a library

• Took 0 min 0 sec . View more details here
• Description: approval-list/elastic/go-sysinfo.yml

Error signal

• Took 0 min 0 sec . View more details here
• Description: githubApiCall: The REST API call https://api.github.com/orgs/elastic/members/redanthrax return the message : java.lang.Exception: httpRequest: Failure connecting to the service https://api.github.com/orgs/elastic/members/redanthrax : httpRequest: Failure connecting to the service https://api.github.com/orgs/elastic/members/redanthrax : Code: 404Error: {"message":"User does not exist or is not a member of the organization","documentation_url":"https://docs.github.com/rest/reference/orgs#check-organization-membership-for-a-user"}

[rvstaveren]

Hi @redanthrax,

A small fix for the Environment stuff. To be sure this also works in a jail I ran the test from there. It works ok, but my shell had an odd environment variabele the test tripped over:

$ go test
Getting Self()...
Getting ProcessInfo...
Getting UserInfo...
Getting Environment...
Getting MemoryInfo...
Getting CPUTimes...
Getting OpenHandleEnumerator...
--- FAIL: TestSelf (0.00s)
    system_test.go:204: 
                Error Trace:    system_test.go:204
...
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -32,3 +32,3 @@
                                  (string) (len=23) "XIM_PROGRAM=ibus-daemon",
                                - (string) (len=19) "XMODIFIERS=@im=ibus",
                                + (string) (len=14) "XMODIFIERS=@im",
                                  (string) (len=19) "_=/usr/local/bin/go"
                Test:           TestSelf
    system_test.go:243: open handles count: 11
FAIL
exit status 1
FAIL    github.com/elastic/go-sysinfo   0.031s
...

fixed by

diff --git a/providers/freebsd/process_freebsd.go b/providers/freebsd/process_freebsd.go
index 117d56c..aa47a53 100644
--- a/providers/freebsd/process_freebsd.go
+++ b/providers/freebsd/process_freebsd.go
@@ -125,7 +125,7 @@ func makeMap(from []string) map[string]string {
        out := make(map[string]string, len(from))
 
        for _, env := range from {
-               parts := strings.Split(env, "=")
+               parts := strings.SplitN(env, "=", 2)
                if len(parts) > 1 {
                        out[parts[0]] = parts[1]
                }

Best Regards, Ruben

[lapo-luchini]

Hope this gets (CLA-signed and) committed soon, as this is a show-stopper for beats 8 support on FreeBSD.

[jurajlutter]

Did this move forward, please?

[redanthrax]

In the year of our lord 2024 I have given up that this will ever be considered.

[vansante]

Yeah, I have also given up and resorted to setting up a personal fork with this merged 🤷

[dbolcsfoldi]

Hello all! I signed the CLA - 2024 will be the year!

[andrewkroh]

I opened https://github.com/elastic/go-sysinfo/pull/204 to add CI testing for FreeBSD.

[lhirlimann]

@redanthrax @dbolcsfoldi Can you guys take a look at the review comments?

[vansante]

I have done some similar things in a fork, feel free to copy some code from here: https://github.com/vansante/go-sysinfo/pull/1/files

Notably the memory sizes I couldnt quite get right without cgo though.

[dbolcsfoldi]

@redanthrax @dbolcsfoldi Can you guys take a look at the review comments?

Sure thing, seems like the only thing needed is to do

git mv providers/freebsd/arch.go providers/freebsd/arch_freebsd.go
git mv providers/freebsd/kernel.go providers/freebsd/kernel_freebsd.go
git mv providers/freebsd/machineid.go providers/freebsd/machineid_freebsd.go

Because this is @redanthrax 's PR I can't push directly to it - for this small of a change I think it makes more sense to just commit to this one with the change?

[andrewkroh]

It looks like there might be some issues with the process data.

• ~cpu user metric is zero~ (9ffae07)
• data from process OpenHandles() seems to have problems (see those empty string)
• ~the process resident_bytes value looks super low (I think ki_rssize needs multiplied by the page size.)~ (60b935f)

{
  "process.cpu": {
    "system": 4450000,
    "user": 0
  },
  "process.env": {
    "BLOCKSIZE": "K",
    "CI": "true",
    "GITHUB_ACTION": "test",
    "GITHUB_ACTIONS": "true",
    "GITHUB_ACTION_REF": "f8be330398166d1eb0601f01353839d4052367b2",
    "USER": "root"
  },
  "process.fd": [
    "/dev/null",
    "",
    "",
    "",
    "",
    "",
    ""
  ],
  "process.info": {
    "args": [
      "/tmp/go-build1363488261/b001/go-sysinfo.test",
      "-test.testlogfile=/tmp/go-build1363488261/b001/testlog.txt",
      "-test.paniconexit0",
      "-test.timeout=10m0s",
      "-test.v=true"
    ],
    "cwd": "/home/runner/work/go-sysinfo/go-sysinfo",
    "exe": "/tmp/go-build1363488261/b001/go-sysinfo.test",
    "name": "go-sysinfo.test",
    "pid": 1619,
    "ppid": 1077,
    "start_time": "2024-04-11T07:18:10.830473Z"
  },
  "process.mem": {
    "resident_bytes": 2022,
    "virtual_bytes": 1273126912
  },
  "process.user": {
    "egid": "0",
    "euid": "0",
    "gid": "0",
    "sgid": "0",
    "suid": "0",
    "uid": "0"
  }
}

[andrewkroh]

Everything but process OpenHandles() appears to be working. This interface is optional. I think we should drop that code and leave it for someone to add in a follow up PR if it is a desired feature. Thoughts?

[jurajlutter]

Everything but process OpenHandles() appears to be working. This interface is optional. I think we should drop that code and leave it for someone to add in a follow up PR if it is a desired feature. Thoughts?

Fine with me.

Btw. by default, a non-root user only see its own file descriptors. Runner does seem to have stdin redirected from /dev/null.

[jurajlutter]

Out of curiosity, how to properly do the tests outside CI?

[andrewkroh]

The standard go test ./... should execute everything in the project.

[jurajlutter]

The standard go test ./... should execute everything in the project.

And to get the output similar to the JSON above?

[jurajlutter]

It looks like there might be some issues with the process data.

• ~cpu user metric is zero~ (9ffae07)
• data from process OpenHandles() seems to have problems (see those empty string)
• ~the process resident_bytes value looks super low (I think ki_rssize needs multiplied by the page size.)~ (60b935f)

{
  "process.cpu": {
    "system": 4450000,
    "user": 0
  },
  "process.env": {
    "BLOCKSIZE": "K",
    "CI": "true",
    "GITHUB_ACTION": "test",
    "GITHUB_ACTIONS": "true",
    "GITHUB_ACTION_REF": "f8be330398166d1eb0601f01353839d4052367b2",
    "USER": "root"
  },
  "process.fd": [
    "/dev/null",
    "",
    "",
    "",
    "",
    "",
    ""
  ],

This looks OK, it only lists open files used by the process itself.

[andrewkroh]

And to get the output similar to the JSON above?

The -v flag triggers verbose output which includes the JSON data. So go test -v ./...

---

This looks OK, it only lists open files used by the process itself.

Why are there empty strings in that list? If that is expected, then perhaps they should be filtered out.

Does the "process.open_handle_count": 10 (log) look correct? The code for OpenHandles() and OpenHandleCount() look similar, but one produces a count of 10 and other a list of length 7.

[jurajlutter]

And to get the output similar to the JSON above?

The -v flag triggers verbose output which includes the JSON data. So go test -v ./...

Yes, I came to it, too :-) Thanks.

This looks OK, it only lists open files used by the process itself.

Why are there empty strings in that list? If that is expected, then perhaps they should be filtered out.

Does the "process.open_handle_count": 10 (log) look correct? The code for OpenHandles() and OpenHandleCount() look similar, but one produces a count of 10 and other a list of length 7.

The logic should be adjusted a bit, I will try to.

[jurajlutter]

And to get the output similar to the JSON above?

The -v flag triggers verbose output which includes the JSON data. So go test -v ./...

Yes, I came to it, too :-) Thanks.

This looks OK, it only lists open files used by the process itself.

Why are there empty strings in that list? If that is expected, then perhaps they should be filtered out. Does the "process.open_handle_count": 10 (log) look correct? The code for OpenHandles() and OpenHandleCount() look similar, but one produces a count of 10 and other a list of length 7.

The logic should be adjusted a bit, I will try to.

The code could read, IMHO:

diff --git a/providers/freebsd/process_freebsd_cgo.go b/providers/freebsd/process_freebsd_cgo.go
index 42dba6a..2dd6df4 100644
--- a/providers/freebsd/process_freebsd_cgo.go
+++ b/providers/freebsd/process_freebsd_cgo.go
@@ -61,10 +61,10 @@ unsigned int countFileStats(struct filestat_list *head) {
 void copyFileStats(struct filestat_list *head, struct filestat *out, unsigned int size) {
   unsigned int index = 0;
   struct filestat *fst;
+  if (size == 0)
+    return;
+
   STAILQ_FOREACH(fst, head, next) {
-    if (!size) {
-      break;
-    }
     memcpy(out, fst, sizeof(*fst));
     ++out;
     --size;
@@ -326,7 +326,7 @@ func (p *process) OpenHandles() ([]string, error) {
        names := make([]string, 0, len(files))

        for _, file := range files {
-               if file.fs_uflags == 0 {
+               if file.fs_type == 1 {
                        names = append(names, C.GoString(file.fs_path))
                }
        }

1. it will not check for size being zero on every iteration over the queue
2. it will check for fs_type instead of fs_uflags

In FreeBSD, a process indeed has open more file descriptors than those referring to actual files and directories.

[jurajlutter]

The networking bits I can review later.

[lhirlimann]

What's left to do here?

[andrewkroh]

/test

[andrewkroh]

What's left to do here?

I think @jurajlutter 's patch needs incorporated to address the empty strings in the OpenHandles() output.

[andrewkroh]

I'm comfortable merging in the current state. I didn't go over the C code too closely, but it's working.

It appears that another reviewer from Elastic is required (probably because I contributed).

[jurajlutter]

What is still left is the network part. I don't have any spare cycles to do the hard work :-(

[andrewkroh]

What is still left is the network part. I don't have any spare cycles to do the hard work :-(

The NetworkCounters can be added in a separate PR (I think that's what you mean). That's an optional part of the provider.