[EPIC] TerriaJS Integration with Magda API

[mwu2018]

When integrated with Magda API to harvest datasets, the following issues need to be addressed.

1. Search datasets (/api/v0/search/datasets) by pagination currently returns inconsistent results. The results may have duplicated datasets and/or miss datasets -- all occur randomly. Although duplicated datasets can be easily filtered out, the missing ones are problematic. The bugs reported in https://github.com/magda-io/magda/issues/2512 and https://github.com/magda-io/magda/issues/2513 might be the cause. The problem will cause dataset number varies with each search. This approach uses only about 20 seconds and provides acceptable UX.
2. There are same name but different identity organisations (about 14) in DGA, which causes many duplicated datasets (about 1800). There is at least one occurrence that datasets have the same source name but with empty or none empty publisher value.
3. Search all organisations (/api/v0/search/organisations) then their datasets (/api/v0/search/datasets) will substantially increase wait time, about 4 - 5 min (not acceptable UX). This approach seems not missing datasets. It will miss datasets without publisher values.
4. Search all organisations then identify none empty groups by fetching only one dataset per group, DOES NOT reduce wait time.
5. Search registry api probably (/api/v0/registry/records) will likely cost more time (bad UX).
6. Terriajs v8 should support wms group, wfs, wfs-group, large geojson etc. Otherwise many harvested datasets can not be visualised. Not all datasets with kml format can be visualised.
7. Magda integration will introduce many datasets. When the number of datasets is large, search UI becomes unresponsive.
8. If a geojson, kml or any other data volume become too large, a browser may become very unresponsive or even crash due to no enough memory -- datasets from data.gov.au need to be downloaded again.