[WIP] Asynchronous acquisition for ATS instruments

[cgranade]

This PR refactors AlazarTech_ATS.acquire to await a new AlazarTech_ATS.async_acquire that runs all ATS API calls in a background thread. This enables running other QCoDeS tasks during an ATS acquisition.

Developed in collaboration with @astafan8.

The main motivation for this PR was that the existing acquire method calls a blocking ATS API call that waits until all buffers are full before returning. This makes it difficult to attach any other logic to an acquisition controller, such as reconfiguring pulse sequences between blocks of buffers. Though one could model that as separate acquisitions, the start-up and tear-down for acquisitions is long enough that it is sometimes desirable to run acquisition in its own background thread. This PR uses a new thread pool executor with a single worker to host all Alazar API calls, then wraps calls out to that thread in async/await pairs to allow the main thread's event loop to proceed with other async tasks while the Alazar thread waits to unblock. Though this is manifestly threadsafe, a new lock is added to guard all Alazar calls to prevent programming errors from causing corruption in the Alazar driver.

[codecov[bot]]

Codecov Report

Merging #1375 into master will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1375   +/-   ##
=======================================
  Coverage   73.86%   73.86%           
=======================================
  Files          92       92           
  Lines       10411    10411           
=======================================
  Hits         7690     7690           
  Misses       2721     2721

[cgranade]

Thanks for the review, @astafan8! I'll edit the description accordingly, then.

[astafan8]

@QCoDeS/core could you provide your feedback on this PR?

[jenshnielsen]

When I try running the synchronous version of the code from one of the standard examples I see

RuntimeError                              Traceback (most recent call last)
<ipython-input-9-21f7ae195c6d> in <module>
----> 1 acquisition_controller.do_acquisition()

c:\users\jens\source\repos\qcodes\qcodes\instrument_drivers\AlazarTech\ATS_acquisition_controllers.py in do_acquisition(self)
     56         """
     57         value = self._get_alazar().acquire(acquisition_controller=self,
---> 58                                            **self.acquisitionkwargs)
     59         return value
     60 

c:\users\jens\source\repos\qcodes\qcodes\instrument_drivers\AlazarTech\ATS.py in acquire(self, mode, samples_per_record, records_per_buffer, buffers_per_acquisition, channel_selection, transfer_offset, external_startcapture, enable_record_headers, alloc_buffers, fifo_only_streaming, interleave_samples, get_processed_data, allocated_buffers, buffer_timeout, acquisition_controller)
    664             allocated_buffers=alloc_buffers,
    665             buffer_timeout=buffer_timeout,
--> 666             acquisition_controller=acquisition_controller
    667         ))
    668 

~\Miniconda3\envs\qcodes\lib\asyncio\base_events.py in run_until_complete(self, future)
    458         future.add_done_callback(_run_until_complete_cb)
    459         try:
--> 460             self.run_forever()
    461         except:
    462             if new_task and future.done() and not future.cancelled():

~\Miniconda3\envs\qcodes\lib\asyncio\base_events.py in run_forever(self)
    412         self._check_closed()
    413         if self.is_running():
--> 414             raise RuntimeError('This event loop is already running')
    415         if events._get_running_loop() is not None:
    416             raise RuntimeError(

RuntimeError: This event loop is already running

I suspect that could be related to the ipython event loop integration. So I tried disabling that with

%autoawait False

but it did not seem to have any effect

[jenshnielsen]

I suspect this is due to https://github.com/jupyter/notebook/issues/3397 as Tornado is already running an eventloop. (The issue was observed in an Jupyter notebook)