Writing a worker
One of the goals of Ocypod is to make writing clients/workers a straightforward process, since all communication takes place using HTTP/JSON.
The simplest possible worker would generally do the following in a loop:
- poll a queue (or queues) for a job to work on
- start processing a job
- when job is complete, update the job's status to
completed
A more fully featured client might do the following:
- poll a queue (or queues) for a job to work on
- start processing a job
- send regular job heartbeat to let Ocypod know the worker is alive
- update the job's
output
field with progress information - when job is complete, update the job's status to
completed
, and update it'soutput
to contain the final result
It's generally assumed that a single client/worker will be updating a job's status, output, etc., but that any number of clients might be reading its metadata (to e.g. update a progress page, to log some statistics, to check for completion, etc.).