JSONOutputFormat has invalid ndarray conversion

[mitar]

Current code:

    def writekvs(self, kvs):
        for k, v in sorted(kvs.items()):
            if hasattr(v, 'dtype'):
                v = v.tolist()
                kvs[k] = float(v)
        self.file.write(json.dumps(kvs) + '\n')
        self.file.flush()

When v has dtype, v is first converted to a list, but then tried to cast to a float.

Moreover, kvs is tried to be modified in place, which would also convert it for any later formaters.

[pzhokhov]

ouch... Thanks for pointing it out!

[Kelvinson]

Hi, I did a small fix and PR -> https://github.com/openai/baselines/pull/836