AI-Horde
AI-Horde copied to clipboard
Stop duplicate images with random seed
I've put a lot of effort in trying to prevent a race condition between workers picking up the same request with the same seed, and therefore returning two identical images.
Each API node gets a random seed at init from the system randomizer and it starts with 0.5 secs delay to avoid thjem getting the same time-based-seed.
random.seed(random.SystemRandom())
Each time a waiting prompt is picked up, I lock its row in the DB, check that it still has n > 0, and only then continue
def start_generation(self, worker):
# We have to do this to lock the row for updates, to ensure we don't have racing conditions on who is picking up requests
myself_refresh = db.session.query(type(self)).filter(type(self).id == self.id, type(self).n > 0).with_for_update().first()
if not myself_refresh:
return None
myself_refresh.n -= 1
db.session.commit()
procgen_class = procgen_classes[self.wp_type]
new_gen = procgen_class(wp_id=self.id, worker_id=worker.id)
self.refresh()
return self.get_pop_payload(new_gen)
And the seed of each job is only set as it's being picked up.
And I still get duplicate generations all too often!
This someone is for someone who suggests a practical solution to prevent this