Outstanding task on client but hub says completed when using broadcast view

[dalbabur]

I want to purge_everything or purge_local_results but I'm getting RuntimeError: Can't purge outstanding tasks: ... The reason I want to purge is that after a couple of tasks my engines memory seems to get full and slows down the process. I'm trying to figure out if it's a memory issue related to ipyparallel or something related to the tasks.

Here is a rough sketch of what I'm doing:

do some setup:

  from ipyparallel import Client, Reference
  rc = Client(client_args, client_kwargs)
  view = rc.broadcast_view(view_args, view_kwargs)
  view.is_coalescing = False
  view.apply_sync( # do some setup stuff )

then:

  ar  = view.apply( # do the real work, and pickle it)
  fvs = np.array(sum([pickle.loads(fv) for fv in ar.get()],[]))

  view.client.purge_local_results('all') # this says there are outstanding tasks
  view.client.queue_status() # this says all tasks are complete