[WIP] CRON like scheduling / scheduler

[treeder]

Triggers functions on a repeating schedule.

Use new input format discussed recently and probably CRON format as well.

---

Proposal for cron support

Basic architecture sketch for each IronFunctions node:

                 ┌────────┐
     ┌──────────▶│D. Lock │
     │           └────────┘
     │
┌────────┐       ┌────────┐
│  Cron  │──────▶│   DB   │◀─────────────┐
└────────┘       └────────┘              │
     │                                   │
     │           ┌────────┐         ┌────────┐
     └──────────▶│   MQ   │◀────────│  HTTP  │
                 └────────┘         └────────┘
                      ▲                  │
                      │                  │
                      │                  ▼
                 ┌────────┐         ┌────────┐
                 │ Async  │         │  Sync  │
                 └────────┘         └────────┘

The two additional components are:

D. Lock is a some sort of distributed lock offered by latest versions of etcd and Consul. In theory, any distributed lock would be enough in order to achieve the necessary role in this design.

Cron is a gear similar to async. It enqueues a cron job while holding a distributed mutex lock. Its implementation would be similar to the following:

lock()
jobs := cronjobs()
for _, job := range jobs {
	ls := laststart(j)
	if job.shouldRunAfter(ls) {
		ls.touch()
		enqueue(j)
	}
}
unlock()

In the DB, the new datastructure would be:

[{
	"id": "...",
	"function": {
		"app": "app",
		"path": "/fn",
		"headers": ["Context-Type: application/json"],
		"body": "....",
	}
	"frequency": "*/10 * * * *"
}, ...]

id is a unique identifier for the cronjob. It aims to allow repeated entries for a same given function.

function is the description of the function to be ran. It will mimic a HTTP POST call with headers and body. It also adds a User-Agent with the specific information that this call is made by the cron worker.

frequency is a crond-like string that specifies when should this cronjob be ran.

Restricting the distributed lock to etcd/consul and alikes, it means they can also be used as an authoritative key-value source of truth regarding the last execution.

laststart in the example code above, is the last time this cronjob was started, regardless of success or failure.