Meta-file synthesizers generating invalid JSON

[btrask]

When we detect links in uploaded Markdown or plain text files, we add those links to the meta-files we generate. However, we don't detect duplicates. Because of the structure of our meta-files (see here), we can get invalid JSON like this:

{
    "link": {
        "http://example.com/": {},
        "http://example.com/": {}
    }
}

Note: We have a good reason for using the structure we do, see the client API link above.

In order to efficiently detect duplicate links, I think we basically need a hash table. This is one of the cases where using C is more painful than a higher level language.

It's worth noting that browsers seem to actually parse this invalid JSON "correctly," replacing the first value with the second value instead of throwing a parse error. Other JSON libraries probably aren't so well-mannered.