XLSX no longer processing due to xlrd

[bushong1]

So it looks like the dependency messytables uses xlrd for excel file processing. The latest xlrd does not support XLSX files anymore due to, as I understand it, security concerns. messytables appears to be a dead project, not having had any activity in the last 2 years. This stack overflow post says that xlrd should be swapped out for openpyxl, but with messytables being unmaintained, that seems unlikely to happen. Is there any effort being taken to support XLSX files?

[anuveyatsu]

@bushong1 there might be some work from our side to fix this but not yet been confirmed. Also, I'd consider replacing Datapusher with Aircan but you'd need to create a new DAG for XLSX loading.

[fishbone1]

When I try to upload an XLSX-file the state remains "pending" forever, which is odd

[categulario]

It seems to me that the option is to replace messytables dependency with its sucesor frictionless

[EricSoroos]

We're also seeing that some .ods files aren't processed well by messytables, essentially causing OOM errors consuming >4G of memory. (among other reasons, it's doing zipfile extraction into memory, and potentially duplicating cells in rows many times to fill a large empty spreadsheet).