Solved: Restarting node server may stop recurring Agenda jobs

Node is the future, it is that simple. With that being said, one of the important things one will look for is how to start cron-jobs, is it by just using cron-tab to start a stand-alone script, or could it be a plugin inside the code base itself, like what is available in Node with its great npm set of packages that you can choose from,
one of the very good packages for managing the cron-jobs is Agenda which comes with a great feature for visualizing your jobs by using Agenda-UI which looks like this:

Problem
After starting using Agenda (0.6.27), I faced a serious issue when restarting my node server, because the recurring jobs (i.e agenda.every '30 minutes') may stop working for no reason, my code was like this:

agenda.start()
agenda.define 'my job', my_job_function
agenda.every '30 minutes', 'my job'

for a while, I thought in leaving Agenda for good, & using the widely known Cron instead, which is a really great alternative by the way, it is almost an imitation of the linux’s cron-tab interface, with an incredible number of downloads (95,483 downloads in the last month),
The only thing kept me trying to find a solution is Agenda’s superior advantage by monitoring the jobs easily using its Agenda-UI interface, so I opened an issue on Agenda’s github page & digged in a little more until I found the solution.

Solution 1
Since redefining our jobs on server start didn’t solve it, so I managed to remove the old broken recurring jobs when shutting down the server like this (you can add the following to your startup scripts like putting it in app.js):

graceful = ()->
    agenda.cancel repeatInterval: { $exists: true, $ne: null }, (err, numRemoved)->
        agenda.stop ()->
            process.exit 0

and with server start, the jobs will be redefined again & voila,
I’m using this workaround now, & it is working like a charm.

Solution 2
While observing the broken jobs & what causes them to stop working, I found that they are locked, because restarting the server while they are still running prevented them from releasing the lock, so droppedoncaprica has proposed the following solution to release all locks when starting the server:

agenda._db.update {lockedAt: {$exists: true } }, { $set : { lockedAt : null } }, (e, numUnlocked)->
    if e
        console.log e
    console.log "Unlocked #{numUnlocked} jobs."
    # redefine your jobs here

Once Agenda solves this issue, I’ll update the post with the version containing the fix isA.