wandb-allennlp icon indicating copy to clipboard operation
wandb-allennlp copied to clipboard

Python yaml package does not parse all float values correctly.

Open dhruvdcoder opened this issue 4 years ago • 2 comments

https://github.com/dhruvdcoder/wandb-allennlp/blob/d36d28775995483e4eb39f227635ff88be1beb1a/wandb_allennlp/allennlp_translator.py#L120

For example, a value 1e-5 will be parsed as a string "1e-5" instead of a float.

Possible solutions:

  1. Have a special regex to parse this before we parse using yaml.load.
  2. You another yaml package like ruamel

How to by-pass the issue without fixing? Use environment variable to get the value to the jsonnet instead of command line override.

Ref: https://stackoverflow.com/questions/30458977/yaml-loads-5e-6-as-string-and-not-a-number

dhruvdcoder avatar Apr 14 '21 04:04 dhruvdcoder

Update: Using environment variable does not work either. Even jsonnet parsers that value as string.

The best way to circumvent this is to make sure that the sweep does not send float values in the e notation.

dhruvdcoder avatar Apr 14 '21 04:04 dhruvdcoder

Update: Using the env variable does not work because even that goes through yaml.load. Moreover, even when the values are not specified in 1e.. notation in the sweep config. The wandb server always sends values in 1e... format when there are more than 5 decimal places. So the only way forward is fixing the yaml load or using json.loads.

dhruvdcoder avatar Apr 19 '21 13:04 dhruvdcoder