Async::Task#stop vs. stop messages?

[postmodern]

Hello, I'm building a unique producer/consumer messaging/job queue using the Async gem, and was wondering what are the pros/cons of calling Async::Task#stop on the main async task to shutdown all workers once all work has been completed vs. sending in a bunch of stop messages into the job queue to get the workers to gracefully exit their consumer loops one-by-one? Is one style preferred over the other with regards to how Async is designed? Do I run the risk of causing deadlocks or exceptions if I call Async::Task#stop?

[ioquatix]

Stopping a task is not graceful, so I suggest you do both. Thread::Queue has a close method which you can use, or you can use Async::Queue with nil if you have single producer or single consumer. I did not introduce Async::Queue#close to keep the implementation simple, but maybe we can if it ends up being convenient.

How you design the "shutdown" part of your program should depend on whether interrupting jobs is acceptable. Normally, using a timer to ensure all work is completed, is reasonable. Closing the work queues, followed by a timeout around a join operation, finally termination, is a good strategy.

[postmodern]

Hmm, I guess I will have to support both styles. I forgot about having to support Ctrl^C and forcibly shutting down the current running jobs instead of waiting for them to finish (which could take a while). However, if all of the workers are idle and there are no jobs currently running (I use JobStarted, JobCompleted, JobFailed messages to track current running jobs) could I safely call .stop on their tasks, or should I still send in some kind of Stop message object which the workers honor?

[ioquatix]

I think sending a shutdown message is a good idea. Your worker should look something like:

while job = queue.pop
  # process job
end

# exit

If multiple workers share the same queue, using Thread::Queue will be better as it has explicit support for #close.