PHRETS
PHRETS copied to clipboard
troydavisson
[2.4] PHP Fatal Error - Memory Exhausted
Version 2.4
CPU Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz (8 core(s))
OS CentOS 6.9 (Final)
Memory 15.57 GB
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 4096 bytes) in /var/www/vhosts/domain.tld/httpdocs/phrets/vendor/troydavisson/phrets/src/Models/Search/Record.php on line 26
With each Search I've noticed the memory is constantly rising using memory_get_peak_usage().
I originally found a work-around by increasing memory_limit to 512M, but now the client wants to include sold properties (essentially tripling the returned results from FlexMLS). Naturally, I can increase the memory_limit, but I just recently updated from 1.x to 2.4. I did not run into the memory issue with the older version.
Update 2017-08-31
I've increased the memory_limit to 1024M with the following fatal error:
PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1 bytes) in /var/www/vhosts/domain.tld/httpdocs/phrets/vendor/troydavisson/phrets/src/Parsers/Search/OneX.php on line 119
Update 2017-08-31 (2)
I've increased the memory_limit to 3G with the following returned log:
Fetching FlexMLS Master Data: CommercialLease Total Records Found: 291 Memory Used: 16.47 MB of 3 GB
Fetching FlexMLS Master Data: CommercialSale Total Records Found: 294 Memory Used: Increased from 16.47 MB to 23.12 MB (+6.65 MB) - 23.12 MB of 3 GB
Fetching FlexMLS Master Data: MultiFamily Total Records Found: 67 Memory Used: Increased from 23.12 MB to 27.36 MB (+4.25 MB) - 27.36 MB of 3 GB
Fetching FlexMLS Master Data: OpenHouse Total Records Found: 137 Memory Used: 27.36 MB of 3 GB
Fetching FlexMLS Master Data: Rental Total Records Found: 791 Memory Used: Increased from 27.36 MB to 53.09 MB (+25.72 MB) - 53.09 MB of 3 GB
Fetching FlexMLS Master Data: Residential Total Records Found: 4418 Memory Used: Increased from 53.09 MB to 301.98 MB (+248.89 MB) - 301.98 MB of 3 GB
Fetching FlexMLS Master Data: VacantLand Total Records Found: 2262 Memory Used: Increased from 301.98 MB to 378.87 MB (+76.9 MB) - 378.87 MB of 3 GB
Fetching FlexMLS Sold Data: CommercialLease Total Records Found: 575 Memory Used: 378.87 MB of 3 GB
Fetching FlexMLS Sold Data: CommercialSale Total Records Found: 404 Memory Used: 378.87 MB of 3 GB
Fetching FlexMLS Sold Data: MultiFamily Total Records Found: 303 Memory Used: 378.87 MB of 3 GB
Fetching FlexMLS Sold Data: Residential Total Records Found: 40326 Memory Used: Increased from 378.87 MB to 2.64 GB (+2.27 GB) - 2.64 GB of 3 GB
Are you sure it's PHRETS causing this and not possibly some outside influence? I had an issue where the framework I'm using, laravel, by default stores all of the previously run queries in memory for debugging and other purposes. This caused each loop in my search, where I'm inserting all of the results from my paginated search, to overrun the memory after about 1000 inserts. Once I figured that out and disabled query logging, I could pull hundreds of thousands of listings and store them without consuming more than 7M, for the entire cycle...even when running for hours continuously like pulling down all records since 2000.
@steveheinsch 1.0rc2 used to run out of memory until I began using FreeResult(). I've checked my logs and see nowhere it is caching the queries. I've updated my issue with more information. For sake of targeting the performance strictly to 2.4, I've disabled all data entry/manipulation and I'm only fetching results.
Hey sir.
We have a problem that can not be solved, so we ask you for help 。
First of all, we have MLS accounts that you got from them. They are:
$rets_login_url = "https://pt.rets.crmls.org/contact/rets/login";
$rets_username = "**FANG";
$rets_password = '**!y$jTv';
I can make sure this account is available，I can make sure this account is available, because I can get the data through this account(That is the basic information Housing)，
I got the data through this phprets 1.x and put it in the CSV, which is what we need。
I see your document is using this method: GetObject ("property", "photo", "*", "1") ,to get the picture.
There are a lot of mlsid fields involved in CSV, but I don't know which field to associate to get the corresponding picture.
So please help me, thank you very much!!!
Good nice to you!!!
------------------ 原始邮件 ------------------ 发件人: "Eric Price";[email protected]; 发送时间: 2017年8月31日(星期四) 晚上9:58 收件人: "troydavisson/PHRETS"[email protected]; 抄送: "Subscribed"[email protected]; 主题: Re: [troydavisson/PHRETS] [2.4] PHP Fatal Error - Memory Exhausted(#167)
Update 2017-08-31
I've increased the memory_limit to 1024M with the following fatal error: PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1 bytes) in /var/www/vhosts/domain.tld/httpdocs/phrets/vendor/troydavisson/phrets/src/Parsers/Search/OneX.php on line 119
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hey sir.
We have a problem that can not be solved, so we ask you for help 。
First of all, we have MLS accounts that you got from them. They are:
$rets_login_url = "https://pt.rets.crmls.org/contact/rets/login";
$rets_username = "**FANG";
$rets_password = '**!y$jTv';
I can make sure this account is available，I can make sure this account is available, because I can get the data through this account(That is the basic information Housing)，
I got the data through this phprets 1.x and put it in the CSV, which is what we need。
I see your document is using this method: GetObject ("property", "photo", "*", "1") ,to get the picture.
There are a lot of mlsid fields involved in CSV, but I don't know which field to associate to get the corresponding picture.
So please help me, thank you very much!!!
Good nice to you!!!
------------------ 原始邮件 ------------------ 发件人: "Eric Price";[email protected]; 发送时间: 2017年9月1日(星期五) 上午7:04 收件人: "troydavisson/PHRETS"[email protected]; 抄送: "Subscribed"[email protected]; 主题: Re: [troydavisson/PHRETS] [2.4] PHP Fatal Error - Memory Exhausted(#167)
@steveheinsch 1.0rc2 used to run out of memory until I began using FreeResult(). I've checked my logs and see nowhere it is caching the queries. I've updated my issue with more information. For sake of targeting the performance strictly to 2.4, I've disabled all data entry/manipulation and I'm strictly just fetching results.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hey sir.
We have a problem that can not be solved, so we ask you for help 。
First of all, we have MLS accounts that you got from them. They are:
$rets_login_url = "https://pt.rets.crmls.org/contact/rets/login";
$rets_username = "**FANG";
$rets_password = '**!y$jTv';
I can make sure this account is available，I can make sure this account is available, because I can get the data through this account(That is the basic information Housing)，
I got the data through this phprets 1.x and put it in the CSV, which is what we need。
I see your document is using this method: GetObject ("property", "photo", "*", "1") ,to get the picture.
There are a lot of mlsid fields involved in CSV, but I don't know which field to associate to get the corresponding picture.
So please help me, thank you very much!!!
Good nice to you!!!
------------------ 原始邮件 ------------------ 发件人: "Steve Heinsch";[email protected]; 发送时间: 2017年9月1日(星期五) 凌晨1:52 收件人: "troydavisson/PHRETS"[email protected]; 抄送: "Subscribed"[email protected]; 主题: Re: [troydavisson/PHRETS] [2.4] PHP Fatal Error - Memory Exhausted(#167)
Are you sure it's PHRETS causing this and not possibly some outside influence? I had an issue where the framework I'm using, laravel, by default stores all of the previously run queries in memory for debugging and other purposes. This caused each loop in my search, where I'm inserting all of the results from my paginated search, to overrun the memory after about 1000 inserts. Once I figured that out and disabled query logging, I could pull hundreds of thousands of listings and store them without consuming more than 7M, for the entire cycle...even when running for hours continuously like pulling down all records since 2000.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hey sir.
We have a problem that can not be solved, so we ask you for help 。
First of all, we have MLS accounts that you got from them. They are:
$rets_login_url = "https://pt.rets.crmls.org/contact/rets/login";
$rets_username = "**FANG";
$rets_password = '**!y$jTv';
I can make sure this account is available，I can make sure this account is available, because I can get the data through this account(That is the basic information Housing)，
I got the data through this phprets 1.x and put it in the CSV, which is what we need。
I see your document is using this method: GetObject ("property", "photo", "*", "1") ,to get the picture.
There are a lot of mlsid fields involved in CSV, but I don't know which field to associate to get the corresponding picture.
So please help me, thank you very much!!!
Good nice to you!!!
------------------ 原始邮件 ------------------ 发件人: "Eric Price";[email protected]; 发送时间: 2017年8月31日(星期四) 晚上9:58 收件人: "troydavisson/PHRETS"[email protected]; 抄送: "Subscribed"[email protected]; 主题: Re: [troydavisson/PHRETS] [2.4] PHP Fatal Error - Memory Exhausted(#167)
Update 2017-08-31
I've increased the memory_limit to 1024M with the following fatal error: PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1 bytes) in /var/www/vhosts/domain.tld/httpdocs/phrets/vendor/troydavisson/phrets/src/Parsers/Search/OneX.php on line 119
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.