dolma icon indicating copy to clipboard operation
dolma copied to clipboard

Some race condition in url taggers

Open peterbjorgensen opened this issue 3 months ago • 1 comments

Even with the latest git version some of the URL taggers crash if I run the taggers with multiprocessing. I can't figure out where this race condition happens. If I run the taggers with --processes 1 first and then do --processes 16 it works.

peterbjorgensen avatar Mar 21 '24 13:03 peterbjorgensen