upgrade plugin structure

[leondz]

desiderata:

1. lazy loading - only load plugins on demand; postpone all heavy ops til the last moment possible
2. fast automatic discovery and enumeration
3. can read metadata without instantiating
4. able to have plugins with conflicting dependencies

consider:

https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/

[jmartin-tech]

Related to #768, #479