Capture all incoming commands in a logfile

[atinmu]

We need something similar to what GD1 provides in cmd_history.log where all incoming CLI commands on that node are logged.

[aravindavk]

incoming API requests are logged(we may have to add parameters as well). CLI logs will not be useful if some functionalities are executed via APIs and some via CLI.

[atinmu]

cmd_history.log file is actually populated by glusterd. What I am requesting for a similar placeholder where one can easily figure out what all commands were (un)successfully executed with a timestamp. Having this captured in glusterd log file itself might not satisfy the need. Believe me, this has immensely helped in debugging different types of problems in glusterd.

[phlogistonjohn]

As @aravindavk noted it makes more sense to log the API calls rather than the cli commands alone.

However, I do have two mad-science ideas. First, command line tools could send the originating cli arguments as part of the rest call, either in a header or in specialized attributes common across the api. If present, the logging would record the original cli that initiated the api in question.

Next, the HTTP requests and responses could be handled by a separate logger/logging endpoint. This endpoint would only contain the requests and responses (something akin to the access_log in apache) I quickly skimmed the logrus docs and it looks like this is doable. However, I think this would only apply in configurations where GD2 logs to files. In a containerized scenario the logging is expected to go to stdio.

[atinmu]

On Fri, Jun 15, 2018 at 11:36 PM, John Mulligan [email protected] wrote:

As @aravindavk https://github.com/aravindavk noted it makes more sense to log the API calls rather than the cli commands alone.

However, I do have two mad-science ideas. First, command line tools could send the originating cli arguments as part of the rest call, either in a header or in specialized attributes common across the api. If present, the logging would record the original cli that initiated the api in question.

Next, the HTTP requests and responses could be handled by a separate logger/logging endpoint. This endpoint would only contain the requests and responses (something akin to the access_log in apache) I quickly skimmed the logrus docs and it looks like this is doable. However, I think this would only apply in configurations where GD2 logs to files. In a containerized scenario the logging is expected to go to stdio.

This would satisfy the requirement. The goal is to capture the requests of the API (with the detailed parameters) and their respective response status somewhere which can be parsed and readable in an easy manner.

—

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gluster/glusterd2/issues/887#issuecomment-397700153, or mute the thread https://github.com/notifications/unsubscribe-auth/AGp7mJA3rvM_r0HVa4JeNFady9dAKOrbks5t8_eYgaJpZM4UoRbP .

[amarts]

I would vote for logging the 'API's. Also, we need a mechanism to identify different clients, like glustercli, gluster-ansible, gluster-csi, gluster-operator etc from API endpoint too. Can this be a Cookie in the header? Should we mandate a Cookie from each client connecting?

[prashanthpai]

Can this be a Cookie in the header?

Yes, a cookie or an identifier. We can also make use of user agent string if present. When using curl, or a web browser, user doesn't have to do anything extra to set it. Example:

$ curl -v http://127.0.0.1:24007/version
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 24007 (#0)
> GET /version HTTP/1.1
> Host: 127.0.0.1:24007
> User-Agent: curl/7.53.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< X-Gluster-Cluster-Id: c27f79c3-8ba0-4664-96a1-2515e91b6d89
< X-Gluster-Peer-Id: 15b7bc28-aa9d-4d89-a0ce-64f73a5b5c3d
< X-Request-Id: 657a4a87-5bd2-4eea-95f1-9b984576bcae
< Date: Tue, 26 Jun 2018 10:13:55 GMT
< Content-Length: 59
< 
{"glusterd-version":"v4.1.0-6.gite010dd6","api-version":1}

Should we mandate a Cookie from each client connecting?

That's not ideal and will be too stringent.

[kshlm]

A logging middleware already exists in the ReST server which logs all incoming requests into the default log file. If required this can be extended to log to a separate file.

And as @prashanthpai mentioned, using a user-agent-string for identifying different types of clients would be the preferred way.

[atinmu]

@phlogistonjohn Would we be able to complete this in a week's time or do you want me to move this to Sprint 2?

[phlogistonjohn]

I have already started on this and should be able to finish it before then.

[atinmu]

Not a blocker for GCS/1.0 with the revised MVP.