Java executors: how to be notified, without blocking, when a task completes?

Say I have a queue full of tasks which I need to submit to an executor service. I want them processed one at a time. The simplest way I can think of is to:

1. Take a task from the queue
2. Submit it to the executor
3. Call .get on the returned Future and block until a result is available
4. Take another task from the queue...

However, I am trying to avoid blocking completely. If I have 10,000 such queues, which need their tasks processed one at a time, I'll run out of stack space because most of them will be holding on to blocked threads.

What I would like is to submit a task and provide a call-back which is called when the task is complete. I'll use that call-back notification as a flag to send the next task. (functionaljava and jetlang apparently use such non-blocking algorithms, but I can't understand their code)

How can I do that using JDK's java.util.concurrent, short of writing my own executor service?

(the queue which feeds me these tasks may itself block, but that is an issue to be tackled later)

From stackoverflow

• Use a CountDownLatch.

  It's from java.util.concurrent and it's exactly the way to wait for several threads to complete execution before continuing.

  In order to achieve the callback effect you're looking after, that does require a little additional extra work. Namely, handling this by yourself in a separate thread which uses the CountDownLatch and does wait on it, then goes on about notifying whatever it is you need to notify. There is no native support for callback, or anything similar to that effect.

  ---

  EDIT: now that I further understand your question, I think you are reaching too far, unnecessarily. If you take a regular SingleThreadExecutor, give it all the tasks, and it will do the queueing natively.

  basszero : or just use Executors.newSingleThreadExecutor()

  Yuval A : I just edited my answer :)

• Define a callback interface to receive whatever parameters you want to pass along in the completion notification. Then invoke it at the end of the task. You could even write a general wrapper for Runnable tasks, and submit these to ExecutorService.

  class CallbackTask implements Runnable {
  
    private final Runnable task;
  
    private final Callback callback;
  
    CallbackTask(Runnable task, Callback callback) {
      this.task = task;
      this.callback = callback;
    }
  
    public void run() {
      task.run();
      callback.complete();
    }
  
  }
  

  Shahbaz : Three answers in the blink of an eye! I like the CallbackTask, such a simple and straight forward solution. It looks obvious in retrospect. Thanks. Regarding others comments about SingleThreadedExecutor: I may have thousands of queues which may have thousands of tasks. Each of them need process their tasks one at a time, but different queues can operate in parallel. That's why I am using a single global threadpool. I'm new to executors so please tell me if I am mistaken.

• If you want to make sure that no tasks will run at the same time then use a SingleThreadedExecutor. The tasks will be processed in the order the are submitted. You don't even need to hold the tasks, just submit them to the exec.

• ThreadPoolExecutor also has beforeExecute and afterExecute hook methods that you can override and make use of. Here is the description from ThreadPoolExecutor's javadocs.

  Hook methods

  This class provides protected overridable beforeExecute(java.lang.Thread, java.lang.Runnable) and afterExecute(java.lang.Runnable, java.lang.Throwable) methods that are called before and after execution of each task. These can be used to manipulate the execution environment; for example, reinitializing ThreadLocals, gathering statistics, or adding log entries. Additionally, method terminated() can be overridden to perform any special processing that needs to be done once the Executor has fully terminated. If hook or callback methods throw exceptions, internal worker threads may in turn fail and abruptly terminate.

  Michael Myers : Don't you love SO's tag-stripping code? I never have figured out a good way to link to any method that takes more than one parameter.

  Cem Catikkas : I'm just giving up... It's becoming more mess as I try to work around.

  Michael Myers : Sometimes I just link to the class docs and say "Somewhere on this page...." :)

  Peter Štibraný : method with percent encoded (, ) and space worked fine ;-)

  Michael Myers : @Peter: Well, at least it *looks* fine. But the anchor links don't work for me in IE7.

  Peter Štibraný : Oh, it works fine in Firefox :-(

  Cem Catikkas : Ah, wonderful!!!

• Or just use a fixed threadpool executor...