Sync no longer terminates

[mohmad-null]

I have a backup script that has been running happily for about 6 months. Several weeks ago I updated the libraries, including Twisted and B2 to the latest versions. (still Python 3.5) Since then, whenever a large file has been synced, the file uploads fine, but the script no longer terminates, it hangs waiting for the large file to finish transferring (it has finished, the content is all there and fine, it just... waits).

Small files are fine.

Has there been some intentional change in the sync code that I'm now misusing, or is this likely to be a regression?

My usage of sync_folders (from b2.sync) looks like this:

	source = parse_sync_folder(pc_source_dir, b2_api)

	# Destination is a bucket
	destination = parse_sync_folder('b2://' + b2_bucket_name, b2_api)

	args = PointlessParameters()

	# Here we capture the output of sys.stdout into temp_std_out
	# This is so we can print to screen and log.
	# We disable progress (no_progress=True) so we don't get lots of progress updates in the log file.
	# Newline = None because otherwise it won't print anything (I think the code is using /r as newline)
	temp_std_out = StringIO(newline=None)
	with redirect_stdout(temp_std_out):
		# Do the syncronisation
		sync_folders(
			source_folder=source,
			dest_folder=destination,
			args=args,
			now_millis=current_time_millis(),
			stdout=sys.stdout,
			no_progress=True,
			max_workers=B2_NUM_THREADS
		)

[bwbeach]

We haven't changed the sync code for quite a while.

What platform are you running on? I would expect the code to use \n for newlines, unless you're running on Windows, where it might use \r\n.

Would it be possible to get a backtrace at the point where it's stuck?

[ppolewicz]

@mohmad-null see how to stop the twisted reactor here, maybe it helps you?

[mohmad-null]

As mentioned, I did update the twisted library as well, so maybe they changed something that introduced this. I think from 16.something to 17.1 which is what I have now.

@bwbeach - I am indeed running this on a Windows platform, though I doubt that the newline thing is a problem (it was there when the code was running fine too). I'm afraid I wouldn't know how to get a backtrace at the point it's sticking ... the code is just hanging there so how would you trigger the backtrace?

@ppolewicz - I've never touched twisted before, what would I do with that?

[ppolewicz]

As far as I've seen in tutorials, Twisted runs an infinite event loop (which hangs forever). You need to shut it down explicitly to terminate your program (as in the link I posted).

[mjurbanski-reef]

Please reopen if problem still occurs with the current version