ReadStat icon indicating copy to clipboard operation
ReadStat copied to clipboard

Reading specific rows from a large `sas7bdat` file

Open BERENZ opened this issue 1 year ago • 1 comments

Is there a way to add functionality to read specific rows from a large sas7bdat file? The issue I'm facing is that I have large SAS files (around 10GB) along with text files (an exact, flat copy of the SAS file). Based on the text file, I can specify the subset of rows that I'm interested in (around 10% of the file).

Another option is to specify a filter while reading, for example, reading rows based on a column. However, I understand that this may be more challenging to implement.

BERENZ avatar Sep 10 '24 09:09 BERENZ

Is there a way to add functionality to read specific rows from a large sas7bdat file? The issue I'm facing is that I have large SAS files (around 10GB) along with text files (an exact, flat copy of the SAS file). Based on the text file, I can specify the subset of rows that I'm interested in (around 10% of the file).

@BERENZ Are readstat_error_t readstat_set_row_offset(readstat_parser_t *parser, long row_offset) and readstat_error_t readstat_set_row_limit(readstat_parser_t *parser, long row_limit) an option here?

Another option is to specify a filter while reading, for example, reading rows based on a column. However, I understand that this may be more challenging to implement.

I do not see how to implement this using the current version of ReadStat.

mettekou avatar Mar 04 '25 14:03 mettekou