BrowserGym
BrowserGym copied to clipboard
Allowing custom backends
Right now, backends are decided by a string that is internally matched with a set of accepted strings:
benchmark = Benchmark(
name="visualwebarena",
backends=["visualwebarena"],
# ...
)
It would be good if we can have a class called Backend that must have a method called prepare, which is called when the backend is being prepared. This would allow anyone to submit a custom backend without having to rewrite the Benchmark class, e.g. backends=[MyBackend()]