Remove full rebuilds for tag indexer with every keystroke or UI click

[pmario]

This PR will fix #7455

As issue #7455 shows, the tag indexes get destroyed with every keystroke in edit mode and with every user UI interaction. IMO that should not be the case.

This PR also contains the performance measurement logic, so Vercel builds a test-version. Once the preview-version is built I'll update the PR and remove that code.

Since the updateDescriptor object is nested and ES5 needs a lot of boilerplate code to avoid "undefined" exceptions I did create several helper variables, which should make the code much more understandable.

As the screenshots at the issue shows, the CSS "rebuilding" speed can be doubled. I did some test with 2000 tags and the relative performance improvement is the same.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
tiddlywiki5	✅ Ready (Inspect)	Visit Preview	Mar 22, 2024 11:46pm

[pmario]

The following text is copied over from #7455

---

I did some initial test with this simple logic, which tests if the tags field exists. No further processing. tiddlywiki.com has 388 tags atm. ... So I would need to test it with more tags 2000 or more.

show code

TagIndexer.prototype.update = function(updateDescriptor) {
	if( (!updateDescriptor.new.tiddler?.fields?.tags && updateDescriptor.old.tiddler === undefined) ||
		(!updateDescriptor.old.tiddler?.fields?.tags && updateDescriptor.new.tiddler === undefined) ||
		(!updateDescriptor.old.tiddler?.fields?.tags && !updateDescriptor.new.tiddler?.fields?.tags)) {
			return;
		} else {
			$tw.utils.each(this.subIndexers,function(subIndexer) {
				subIndexer.update(updateDescriptor);
			});
		}
};

Then I did measure, how long a rebuild takes.

show code

TagSubIndexer.prototype.rebuild = function() {
	var self = this;
	// Hashmap by tag of array of {isSorted:, titles:[]}
const t0 = $tw.utils.timer();
	this.index = Object.create(null);
	// Add all the tags
	this.indexer.wiki[this.iteratorMethod](function(tiddler,title) {
		$tw.utils.each(tiddler.fields.tags,function(tag) {
			if(!self.index[tag]) {
				self.index[tag] = {isSorted: false, titles: [title]};
			} else {
				self.index[tag].titles.push(title);
			}
		});
	});
console.log("--------------- rebuild", $tw.utils.timer(t0));
};

The TW startup shows this. Every rebuild on my machine needs between 1ms and 3ms (worst case)

That's not as much as I thought, but every "styleRefresh" needs about 2.5ms now. That is about 2ms better with basically every click on the UI.

v5.2.7 code needs about double the time.

---

My machine is an Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz 3.60 GHz ... Not the newest machine but probably 10 times faster than an average mobile device.

[linonetwo]

Great, in most of the times, there is no new tiddler will be the case.

[pmario]

Great, in most of the times, there is no new tiddler will be the case.

I see what you mean. I'll ad an extra if (noNewTid) return after the first variable definitions to avoid the a,b,c evaluation

[pmario]

@linonetwo .. I did implement the "return early" logic and it works for me. Can you test and also make some measurements. If it makes a difference for your projects?

The code I used for measurements is documented at: https://github.com/Jermolene/TiddlyWiki5/pull/7615#issuecomment-1642078800 behind the "show code buttons"

[linonetwo]

I don't know where I can test it better than you, I think code is good, I can put the prerelease into TidGi after this is merged.

[Jermolene]

Hi @pmario looking at this again, I think the logic is faulty. The function bails early if it receives a deletion, which seems wrong. Let's look at this again after v5.3.2

[pmario]

converted this PR into a draft, since it contains console.log() code, which should not be part of the PR

[pmario]

The ZIP file contains 2 single-file wikis.

• tag-index.html contains the "old code" + console.log() - which updated the tag-caches with every keystroke.

• tag-optimized.html contains the "new code" + console.log() - but it avoids tag-cache updates for most UI interactions.

• both wikis contain 20000 new tiddlers with 20000 new tags

  • WARNING If you open the Right Sidebar -> More -> Tags tab it may brick your browser.
  • It needs about 12 seconds on my machine.

tag-indexer-performance-measurements.zip

The best way to see the differences is, to open the browser dev-tools with F12 and interact with the UI.

• tag-index.html will activate a lot more cache-updates
• styleRefresh will need about 90% more time with every interaction
• At browser reload tag-index.html on my machine needs about 80ms more time because of unnecessary cache refreshes