Possible race condition in ContentItemInstaller

[vitaliitylyk]

I have noticed that for some reason Delete local items not in remote server functionality removes extra items. I have run content sync between 2 Sitecore instances with exactly the same content. After sync had been finished I noticed that in the first instance some item trees were missing. So I started digging into code and noticed the following piece in ContentItemInstaller:

using (new SecurityDisabler())
using (new SyncOperationContext())
{
	while (!Completed)
	{
		if (!itemsToInstall.TryTake(out var remoteData, int.MaxValue, cancellationToken))
		{
			break;
		}

In case itemsToInstall.TryTake(out var remoteData, int.MaxValue, cancellationToken) returns false, the installer stops installing and eventually CleanUnwantedLocalItems kicks in. This code can return false when collection is empty.

I might be wrong, but how does the ContentItemInstaller know that ContentItemPuller has finished pulling? I cannot find any reference to ContentItemPuller.Completed property. It seems to me that when itemsToInstall becomes empty then ContentItemInstaller will stop installing even though ContentItemPuller might eventually put more items in the collection?

[JeffDarchuk]

itemsToInstall is a BlockingCollection. If the collection is empty and the thread adding things to the BlockingCollection hasn't switched the CompleteAdding flag it'll thread sleep until an item is added to the collection. If the collection is empty and the CompleteAdding flag is set then it'll return false in the TryTake method. You can see how it's being passed along Here

That being said, missing trees of items is definitely a problem. Are you using Item Blaster? Are you using the latest version 1.5.9?

[vitaliitylyk]

Okay, I see, so this works due to specifics of BlockingCollection. Thanks for explanation!

Yes, I am using Item Blaster, Sidekick version is 1.5.9. I can try to do couple of syncs without Blaster enabled and see if it makes any impact.

[JeffDarchuk]

yeah, that would be a good test. Additionally if your situation has anything notable about it that might help me reproduce this, please share.

[markgibbons25]

This would explain part of the issue I've seen in #58 where I'm missing trees, perhaps partial trees too can be explained by this. @JeffDarchuk , one thing of note I can think of that is relevant is that there is some latency between Production and the local machine - up to 200ms

[markgibbons25]

I downgraded to 1.5.2 and this problem went away. So I would say there is a bug here that you can reproduce. All my test cases is using a remote server.

[markgibbons25]

I haven't seen this one in a long time since I set databaseWriterThreads to 1 as per #63