Slow start due to quadratic time algorithms

[purplesyringa]

Step 1: Please describe your environment

• ZeroNet version: v0.7.2 (rev4555)
• Operating system: N/A
• Web browser: N/A
• Tor status: N/A
• Opened port: N/A
• Special configuration: N/A

Step 2: Describe the problem:

ZeroNet takes about an hour to start with a few thousand sites downloaded.

The logs say:

[13:03:17] SiteManager Added new site: [address]
[13:03:17] Site:1A28C5..E1dw ContentDb init: 0.000s, found files: 0, sites: 8963
[13:03:17] Site:1A28C5..E1dw ContentDb not initialized, load files from filesystem...
[13:03:17] Site:1A28C5..E1dw Content.json not exist: ./data/[address]/content.json
[13:03:17] RateLimit Added to queue (2.75s left): Save sites.json 
[13:03:17] SiteManager Updated merger sites in 0.601s

(repeat a few thousand times)

The 'Updated merger sites' seems to be the culprit. It's from SiteManagerPlugin.updateMergerSites, which scans all the downloaded sites, and is invoked every time a site is initialized. That takes quadratic time.

Steps to reproduce:

1. Create/download lots of sites
2. ???
3. PROFIT

Observed Results:

Quadratic time intiailization.

Expected Results:

Linear (ideally) time initialization.

[purplesyringa]

This patch seems to solve the problem for me somewhat

diff --git a/plugins/MergerSite/MergerSitePlugin.py b/plugins/MergerSite/MergerSitePlugin.py
index 2dccc6de..5c706867 100644
--- a/plugins/MergerSite/MergerSitePlugin.py
+++ b/plugins/MergerSite/MergerSitePlugin.py
@@ -335,6 +335,11 @@ class SitePlugin(object):
             for ws in merger_site.websockets:
                 ws.event("siteChanged", self, {"event": ["file_failed", inner_path]})
 
+    def saveSettings(self, *args, **kwags):
+        super(SitePlugin, self).saveSettings(*args, **kwags)
+        if "content.json" in self.content_manager.contents:
+            site_manager.updateMergerSites()
+
 
 @PluginManager.registerTo("SiteManager")
 class SiteManagerPlugin(object):
@@ -393,7 +398,3 @@ class SiteManagerPlugin(object):
     def load(self, *args, **kwags):
         super(SiteManagerPlugin, self).load(*args, **kwags)
         self.updateMergerSites()
-
-    def saveDelayed(self, *args, **kwags):
-        super(SiteManagerPlugin, self).saveDelayed(*args, **kwags)
-        self.updateMergerSites()