postgresql-cursor

postgresql-cursor copied to clipboard

= PostgreSQLCursor

PostgreSQL Cursor is an extension to the ActiveRecord PostgreSQLAdapter for very large result sets. It provides a cursor open/fetch/close interface to access data without loading all rows into memory, and instead loads the result rows in "chunks" (default of 10_000 rows), buffers them, and returns the rows one at a time.

For web pages, an application would not want to process a large amount of data, usually employing a Pagination scheme to present it to the users. Background processes sometimes need to generate a large

Previous solutions employ pagination to fetch each block, then re-running the query for the next "page". This plugin avoids re-executing the query by using the PostgreSQL cursors.

Like the #find_by_sql method, #find_cursor returns each row as a hash instead of an instantiated model class. The rationale for this is performance, though an option to return instances is available. Julian's benchmarks showed returning instances was a factor of 4 slower than return the hash.

A Rails/ActiveRecord plugin for the PostgreSQL database adapter that will add cursors to a find_cursor() method to process very large result sets.

the find_cursor method uses cursors to pull in one data block (of x records) at a time, and return each record as a Hash to a procedural block. When each data block is exhausted, it will fetch the next one.

find_cursor_by_sql takes a custom SQL statement and returns each row.

==Example

Account.find_cursor(:account_cursor, true, :all, :conditions=>["status = ?", 'good']) do |row| puts row.to_json end

Account.find_cursor_by_sql(:account_cursor, "select ...", *args) do |row| puts row.to_json end

==Authors Allen Fair, [email protected], http://github.com/afair

Thank you to:

• Iulian Dogariu, http://github.com/iulianu (Fixes)
• Julian Mehnle, http://www.mehnle.net (Suggestions)

Copyright (c) 2010 Allen M. Fair, released under the MIT license