Async requests are randomly failing

[mxr576]

Super dummy script that confirms this behaviour:

curl.php

<?php

require_once "vendor/autoload.php";

use GuzzleHttp\Promise;
use GuzzleHttp\Psr7\Request;

$pluginClient = new \Http\Client\Common\PluginClient(
  new \Http\Client\Curl\Client(\Http\Discovery\MessageFactoryDiscovery::find(), \Http\Discovery\StreamFactoryDiscovery::find())
);

$iteration = 1;
$exit = 0;
while (TRUE) {
  if ($iteration > 20) {
    echo('Passed' . PHP_EOL);
    break;
  }

// Initiate each request but do not block
$promises = [
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/500')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/404')),
  $pluginClient->sendAsyncRequest(new Request('GET', 'http://httpbin.org/status/200')),
];

  try {
    // Wait for the requests to complete, even if some of them fail
    $results = Promise\settle($promises)->wait();
    foreach ($results as $result) {
      if ($result['state'] === Promise\Promise::REJECTED && is_string($result['reason'])) {
        printf("Iteration: %d. %s\n", $iteration, $result['reason']);
        $exit = 1;
        break 2;
      }
    }
    $iteration++;
  }
  catch (\Exception $e) {
    printf("Exception should not be thrown. %s\n", get_class($e));
    $exit = 1;
    break;
  }
}

exit($exit);

testrunner.sh

#!/usr/bin/env bash

COUNTER=1;
while true; do
    php curl.php
    if [ $? -eq 1 ]; then
        echo Failed after: $COUNTER restarts.;
        break
    fi
    let COUNTER=COUNTER+1
    sleep 1
done

Outputs:

[email protected]:/var/www/html $ ./testrunner.sh 
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Iteration: 1. Invoking the wait callback did not resolve the promise
Failed after: 10 restarts.
[email protected]:/var/www/html $ ./testrunner.sh 
Iteration: 1. Invoking the wait callback did not resolve the promise
Failed after: 1 restarts.

PHP version: PHP 7.1.17 (cli) (built: Apr 27 2018 07:21:42) ( NTS ) Copyright (c) 1997-2018 The PHP Group Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies with Zend OPcache v7.1.17, Copyright (c) 1999-2018, by Zend Technologies with Xdebug v2.6.0, Copyright (c) 2002-2018, by Derick Rethans I have tested this with enabled and disabled xDebug.

As you can see the tester either fails in the first iteration or it does not fail at all.

[mxr576]

Same with PHP 7.2

[email protected]:/var/www/html $ ./testrunner.sh 
Passed
Iteration: 1. Invoking the wait callback did not resolve the promise
Failed after: 2 restarts.
[email protected]:/var/www/html $ php -v
PHP 7.2.5 (cli) (built: May 22 2018 07:51:39) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.5, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.6.0, Copyright (c) 2002-2018, by Derick Rethans

[mxr576]

I tried to confirm that this issue did not cause by some I/O problem in Docker containers by using Docker for Mac. It took more time to fail, but it failed.

➜  web git:(async) ✗ ./testrunner.sh
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Passed
Iteration: 1. Invoking the wait callback did not resolve the promise
Failed after: 23 restarts.

➜  web git:(async) ✗ php -v
PHP 5.6.30 (cli) (built: Oct 29 2017 20:30:32) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies

[joelwurtz]

Being only integration 1 seems normal, since you never restart fetching request. IMO you have this error when a request fail from the backend (network error ?) and the error message is not very explicit or the error behavior is badly handled

[mxr576]

Right, updated to POC code in the 1st comment.

I spin up an Ubuntu 18.04 with PHP 7.2 in a VM just to completely exclude Docker issues from the picture. It was much complicated to reproduce this issue but I still managed to do that.

[mxr576]

If you add ErrorPlugin to the Plugin client it becomes much easier to reproduce this bug:

$pluginClient = new \Http\Client\Common\PluginClient(
  new \Http\Client\Curl\Client(\Http\Discovery\MessageFactoryDiscovery::find(), \Http\Discovery\StreamFactoryDiscovery::find()), [
    new \Http\Client\Common\Plugin\ErrorPlugin()
  ]
);