AI-Horde icon indicating copy to clipboard operation
AI-Horde copied to clipboard

Stop duplicate images with random seed

Open db0 opened this issue 1 year ago • 0 comments

I've put a lot of effort in trying to prevent a race condition between workers picking up the same request with the same seed, and therefore returning two identical images.

Each API node gets a random seed at init from the system randomizer and it starts with 0.5 secs delay to avoid thjem getting the same time-based-seed.

random.seed(random.SystemRandom())

Each time a waiting prompt is picked up, I lock its row in the DB, check that it still has n > 0, and only then continue

    def start_generation(self, worker):
        # We have to do this to lock the row for updates, to ensure we don't have racing conditions on who is picking up requests
        myself_refresh = db.session.query(type(self)).filter(type(self).id == self.id, type(self).n > 0).with_for_update().first()
        if not myself_refresh:
            return None
        myself_refresh.n -= 1
        db.session.commit()
        procgen_class = procgen_classes[self.wp_type]
        new_gen = procgen_class(wp_id=self.id, worker_id=worker.id)
        self.refresh()
        return self.get_pop_payload(new_gen)

And the seed of each job is only set as it's being picked up.

And I still get duplicate generations all too often!

This someone is for someone who suggests a practical solution to prevent this

db0 avatar Mar 07 '23 22:03 db0