BrowserGym icon indicating copy to clipboard operation
BrowserGym copied to clipboard

Allowing custom backends

Open xhluca opened this issue 8 months ago • 0 comments

Right now, backends are decided by a string that is internally matched with a set of accepted strings:

benchmark = Benchmark(
    name="visualwebarena",
    backends=["visualwebarena"],
    # ...
)

It would be good if we can have a class called Backend that must have a method called prepare, which is called when the backend is being prepared. This would allow anyone to submit a custom backend without having to rewrite the Benchmark class, e.g. backends=[MyBackend()]

xhluca avatar Feb 17 '25 22:02 xhluca