-
Notifications
You must be signed in to change notification settings - Fork 223
Description
Hey there,
Thanks for the great library, although unfortunately I have been having a few issues,
I've been using bull and recently switched to bee queue to make use of the job success event callbacks that mean I can use async.js parallel to set of loads of tasks to be done by workers and wait for them all to complete,
My problem comes when the queue says that it has "completed" everything
{ waiting: 0,
active: 0,
succeeded: 1250,
failed: 0,
delayed: 0,
newestJob: 1250 }But by my own logging:
Checked stalled jobs 0
Total: 1250 jobs
Outstanding: 10 jobs
10 Jobs didn't fire a callback, which then hangs my program which is waiting for the success callback of each job before it moves on to the next step
This is probably a scalability issue of where jobs are coming back to the dispatch too quickly to handle, as when jobs have a setTimeout in them of 100ms the queue doesn't hang. But in that case, I would expect for succeeded to be incorrect when calling checkHealth (although I know this is asking Redis instead of the library what it has seen).
As all jobs have "succeeded" is there a way for me to then recall the data returned by these outstanding jobs? Or is there a way for me to forcefully retry them again?
For now I'm going to attempt to set a manual timeout to remove and save the job again.
Thanks for the hard work though, cheers for any help anyone may have for fixing this,