readthedocs.org icon indicating copy to clipboard operation
readthedocs.org copied to clipboard

Telemetry: verify `pip list --format json` fields

Open humitos opened this issue 2 years ago • 0 comments

In some situations we are receiving an extra metadata field with a lot of data when calling pip list --format json here https://github.com/readthedocs/readthedocs.org/blob/7e0fa4ea272bb700119623b163a9cf93babdfa25/readthedocs/telemetry/collectors.py#L124-L160

In [11]: BuildData.objects.get(pk=151444).data["packages"]["pip"]["user"][0]
Out[11]: 
{'name': 'furo',
 'version': '2022.6.4.1',
 'metadata': {'name': 'furo',
  'summary': 'A clean customisable Sphinx documentation theme.',
  'version': '2022.6.4.1',
  'classifier': ['Framework :: Sphinx',
   'Framework :: Sphinx :: Theme',
   'Development Status :: 5 - Production/Stable',
   'License :: OSI Approved :: MIT License',
   'Environment :: Web Environment',
   'Intended Audience :: Developers',
   'Programming Language :: Python :: 3',
   'Programming Language :: Python :: 3.5',
   'Programming Language :: Python :: 3.6',
   'Programming Language :: Python :: 3.7',
   'Programming Language :: Python :: 3.8',
   'Programming Language :: Python :: 3.9',
   'Operating System :: OS Independent',
   'Topic :: Documentation',
   'Topic :: Software Development :: Documentation'],
  'description': '<h1 align="center">Furo</h1>\n<p align="center">\n  A clean customisable <a href="https://www.sphinx-doc.org/">Sphinx</a> documentation theme.\n</p>\n<a href="https://pradyunsg.me/furo/">\n  <img align="center" src="https://github.com/pradyunsg/furo/raw/main/docs/_static/demo.png" alt="Demo image">\n</a>\n\n## Elevator pitch\n\n<!-- start elevator-pitch -->\n\n- **Intentionally minimal** --- the most important thing is the content, not the scaffolding around it.\n- **Responsive** --- adapting perfectly to the available screen space, to work on all sorts of devices.\n- **Customisable** --- change the color palette, font families, logo and more!\n- **Easy to navigate** --- with carefully-designed sidebar navigation and inter-page links.\n- **Good looking content** --- through clear typography and well-stylised elements.\n- **Good looking search** --- helps readers find what they want quickly.\n- **Biased for smaller docsets** --- intended for smaller documentation sets, where presenting the entire hierarchy in the sidebar is not overwhelming.\n\n<!-- end elevator-pitch -->\n\n## Quickstart\n\n<!-- start quickstart -->\n\nFuro is distributed on [PyPI]. To use the theme in your Sphinx project:\n\n1. Install Furo in documentation\'s build environment.\n\n   ```text\n   pip install furo\n   ```\n\n2. Update the `html_theme` in `conf.py`.\n\n   ```py\n   html_theme = "furo"\n   ```\n\n3. Your Sphinx documentation\'s HTML pages will now be generated with this theme! 🎉\n\n[pypi]: https://pypi.org/project/furo/\n\n<!-- end quickstart -->\n\nFor more information, visit [Furo\'s documentation][quickstart-docs].\n\n[quickstart-docs]: https://pradyunsg.me/furo/quickstart\n\n## Contributing\n\nFuro is a volunteer maintained open source project, and we welcome contributions of all forms. Please take a look at our [Contributing Guide](https://pradyunsg.me/furo/contributing/) for more information.\n\n## Acknowledgements\n\nFuro is inspired by (and borrows elements from) some excellent technical documentation themes:\n\n- [mkdocs-material] for MkDocs\n- [Just the Docs] for Jekyll\n- [GitBook]\n- [pdoc3]\n\nWe use [BrowserStack] to test on real devices and browsers. Shoutout to them for supporting OSS projects!\n\n[mkdocs-material]: https://squidfunk.github.io/mkdocs-material/\n[just the docs]: https://just-the-docs.github.io/just-the-docs/\n[gitbook]: https://www.gitbook.com/\n[pdoc3]: https://pdoc3.github.io/pdoc/doc\n[browserstack]: https://browserstack.com/\n\n## What\'s with the name?\n\nI plucked this from the scientific name for [Domesticated Ferrets](https://en.wikipedia.org/wiki/Ferret): Mustela putorius **furo**.\n\nA ferret is actually a really good spirit animal for this project: cute, small, steals little things from various places, and hisses at you when you try to make it do things it doesn\'t like.\n\n> I plan on commissioning a logo for this project (or making one myself) consisting of a cute ferret. Please reach out if you\'re interested!\n\n## Used By\n\n<!-- start used-by -->\n\n> I\'m being told that mentioning who uses `$thing` is a good way to promote `$thing`.\n\n- [urllib3] -- THE first adopter of Furo\n- [attrs] -- one of the early adopters!\n- [pip] -- what I wrote this for\n- [Python Developer’s Guide][devguide]\n- [psycopg3] -- another of the early adopters!\n- [black]\n\n[urllib3]: https://urllib3.readthedocs.io/\n[attrs]: https://www.attrs.org/\n[devguide]: https://devguide.python.org/\n[pip]: https://pip.pypa.io/\n[psycopg3]: https://www.psycopg.org/psycopg3/docs/\n[black]: https://black.readthedocs.io/en/stable/\n\n<!-- end used-by -->\n\n## License\n\nThis project is licensed under the MIT License.\n',
  'project_url': ['Github, https://github.com/pradyunsg/furo'],
  'author_email': 'Pradyun Gedam <[email protected]>',
  'requires_dist': ['beautifulsoup4',
   'sphinx<6.0,>=4.0',
   'sphinx-basic-ng',
   'pygments'],
  'requires_python': '>=3.7',
  'metadata_version': '2.1',
  'description_content_type': 'text/markdown'}}

This is storing a lot of data we don't need. We should only grab the keys we are expecting name and version.

humitos avatar Aug 03 '22 09:08 humitos