OrientDB-NET.binary icon indicating copy to clipboard operation
OrientDB-NET.binary copied to clipboard

Implement an OrientDataReader

Open tbertels opened this issue 8 years ago • 3 comments

When querying a lot of data, using a DataReader is useful and sometimes necessary to avoid running out of memory.

Here's some information about it:

Retrieving Data Using a DataReader IDataReader Interface

That kind of functionality exists on the server side: Pagination · OrientDB Manual

I don't know if it works, but there seems to be one in Java: https://github.com/svn2github/SpagoBI-V4x/blob/master/SpagoBIUtils/src/it/eng/spagobi/tools/dataset/common/datareader/JDBCOrientDbDataReader.java

Npgsql, an open source .NET database driver for PostgreSQL, does it very nicely, you can check it out for an example of implementation.

Here's an example of use:

public void GetAllVertices()
{
    int numberOfVertices = 0
    OrientConnection connection;
    connection = new OrientConnection("localhost", 2424, "MyDB", ODatabaseType.Graph, "root", "azer");
    OrientCommand command = new OrientCommand("SELECT FROM V", connection);
    connection.Open();

    OrientDataReader reader = command.ExecuteReader();

    if (reader.HasRows)
    {
        while (reader.Read())
        {
            Console.WriteLine(reader["vertexID"].ToString()); // "vertexID" is a vertex property
            numberOfVertices++;
        }
        Console.WriteLine("Total vertices count: " + numberOfVertices);
    }
    else
    {
        Console.WriteLine("No vertex found.");
    }
    reader.Close();
}

tbertels avatar Dec 23 '16 11:12 tbertels

I don't understand the issue. Could you please provide more information on the problem you are having?

GrayDelacluyse avatar Dec 23 '16 16:12 GrayDelacluyse

I've added an example of use and a link to the functionality on the server side. A DataReader makes it easy to retrieve data from a large query without running out of memory.

tbertels avatar Dec 23 '16 17:12 tbertels

+1. The issue is that all of the results of a queries are loaded into memory (ie, ToList()) at execution. This is fine for trivial data sets, but not practical for large sets. The paging described in the link is a workaround, but ideally a developer would like to foreach through the results, as you would in a traditional DataReader.

bernierm avatar Mar 22 '17 18:03 bernierm