Skip deleted observations in SAS7BDAT files

[peiyaoli2]

Hi, it seems like ReadStat also has a similar issue as https://github.com/pandas-dev/pandas/issues/15963, where deleted observations are still being counted in rows & being read. Here is a sample output from ReadStat on the same file as the other issue:

readstat /Users/peiyao.li/Downloads/datetime.sas7bdat
Format: SAS data file (SAS7BDAT)
Columns: 5
Rows: 5
Table name: DATETIME
Format version: 8
Text encoding: WINDOWS-1252
Byte order: little-endian
Timestamp: 08 Apr 2017 09:51

And the screenshot from the other thread is shown that there are 3 rows and 2 deleted observations.

[peiyaoli2]

@evanmiller can you take a look at this? Thanks!

[evanmiller]

Hi, I would need a sample file to address this issue.

[evanmiller]

It looks like this code may help

https://github.com/troels/pandas/commit/b31144ac706b9386dc4ff165f69e950fb72e6aa0

[peiyaoli2]

Hi! Here is the datetime.sas7bdat used to repro this: datetime.sas7bdat.zip

It's the same file as the https://github.com/pandas-dev/pandas/issues/15963 issue

[evanmiller]

@peiyaoli2 Thanks for the link to the file. Based on the code that I linked, it looks like skipping deleted rows will be a bit of work. I'll leave this issue open, but I'm going to change it from a "bug" to an "enhancement".

[peiyaoli2]

@evanmiller Hi! I would like to know if there is any ETA on when this feature can be supported. Thanks!

[ito4natsf]

@evanmiller Hi. My client is facing this issue and needs an immediate fix. So I'd also like to know if there is any ETA for this to be implemented. Thanks and regards.

[ito4natsf]

@evanmiller Hi, sorry for bothering you as I send you again, but could you provide ETA for this to be implemented?

[evanmiller]

No ETA

[ito4natsf]

@evanmiller Could you please implement this enhancement, since my customer using readstat repeatedly asks me to do that?