AngryCurl icon indicating copy to clipboard operation
AngryCurl copied to clipboard

Add method to clear Request List

Open weblooter opened this issue 6 years ago • 0 comments

When used in a loop, there is a problem that Get/Post/Request adds new links to the list, but they are not discarded after execute.

static::$obHttpClient = new \AngryCurl(function ($strResponse, $arInfo, $obRequest)
            {
                return static::loadPage($strResponse, $arInfo, $obRequest);
            });
        if ($debugMode) {
            static::$obHttpClient->init_console();
        }
        static::$obHttpClient->load_proxy_list(static::getProxyList());
        static::$obHttpClient->load_useragent_list(static::getUserAgentList());

        while (!empty(static::$arUrlQueue)) {
            static::$obHttpClient->clearRollings();
            foreach (static::$arUrlQueue as $strLink => $intLeftTryConnect) {
                static::$obHttpClient->get($strLink);
            }
            static::$obHttpClient->execute(2);
        }

If we have groups of 10 links, then after the fifth cycle of links will be already 50 (10, 20, 30, 40, 50)

Solution - clean \RollingCurl::$requests after each execute(); Or add method to \RollingCurl

public function clearRollings()
    {
        $this->requests = [];
    }

Please do, because the library is very good.

weblooter avatar Jul 21 '19 14:07 weblooter