Callback after complete crawl

[grgaortiz]

Is there a way to get a callback once all the queued crawls are complete instead of returning per crawl? For example, callback with all url/email pairs instead of one-by-one. If not, do you know if a way to do this with the current callback structure using async?

[0xMeir]

the code here is just regex to find emails var body = $("body").html(); if (body) { (body.match(/([a-zA-Z0-9.-]+@[a-zA-Z0-9.-]+.[a-zA-Z0-9._-]+)/gi) || []).forEach(function(email){ if (callback) callback(result.options.uri,email); });
}

If you dont need the database stuff (if you are not crawling huge websites you probably don't) then just implement this manually, and instead of doing a callback add each email to an array. otherwise just modify the code here.