mesos
How can I obtain current job state and slave where it is executed via REST API?
Hi, I need to obtain the info about the state of the job and the slave that is executing the job (if proceed) to use it inside a python program. I have used the command provided in the Chronos documentation (curl -L -X GET chronos-node:8080/scheduler/jobs) to obtain the info about current jobs in Chronos 2.4.0 via its REST API, but information I need is not provided. I have read that it is possible to obtain those data using Cassandra (http://wjb-tech.blogspot.com.es/), but I was not willing to install it. Is there any way to obtain this info without installing Cassandra?
Thanks!
I found that:
curl -L -X GET http://chronos-node:4400/scheduler/graph/csv
obtains the current state of the job. Nothing about the node...
To get the state in a json format you can use something like the following:
curl -X GET https://chronos-node:8080/v1/scheduler/jobs/summary
Unfortunately that leaves out most of the data from the regular /v1/scheduler/jobs/ api, so a combination of the two would be needed, joined together on the job names, in order to get the job with the state included.
To get the current state, if you have access to the Mesos leader endpoint, and your job is named 'myjob', you can use this Mesos /master/tasks API do something like this:
curl -s -X GET http://mesos-leader-node:5050/master/tasks | sed $'s|},{|},\\\n{|g' | grep '"ChronosTask:myjob"'
Use the nested quotes on the grep argument to make sure that the job name is literally 'myjob' and does not just start with that. The chronos framework creates the mesos task as "ChronosTask:<chronosjobname>", which makes it searchable within the mesos state if each task is placed on its own line by the sed command. If you're doing it in python you could parse the entire return from /master/tasks into a json object and search through that for the task name that matches, or if you're doing it as a subprocess call, you can just sed/grep it before it gets parsed by python, which is almost certainly better performance.
In the JSON value this returns, it gives you the 'slave_id', 'framework_id', and the 'state' as fields. The state here is the mesos task state, which does not directly match up with the Chronos job state returned by the first curl command.
If you need the hostname for the mesos slave that ran that chronos job you can use the /master/slaves API, but unfortunately it seems like the slave_id query param is not working to filter down to the particular slave, so you'll have to filter it manually again. For this one there isn't a huge return value so you can filter it just in python.
curl -s -X GET http://mesos-leader-node:5050/master/slaves
And in the 'slaves' field of the json dict it returns, there is a list of slave objects, each with the 'id' and 'hostname' fields.
