pyam icon indicating copy to clipboard operation
pyam copied to clipboard

Consider replacing regex for matching * with fnmatch

Open phackstock opened this issue 3 years ago • 3 comments

Fnmatch is a python standard library tool which provides unix style wildcard pattern matching. Not as powerful as regex of course but for just matching with * it should be sufficient.

Example for finding all .csv files in a folder:

import fnmatch
import os

for file in os.listdir('.'):
    if fnmatch.fnmatch(file, '*.csv'):
        print(file)

This could replace using regex per default for the pyam filter function.

phackstock avatar Jun 01 '21 08:06 phackstock

Thanks for the suggestion @phackstock - this could be a nice way to get rid of the utils function _escape_regexp() within pattern_match() (for the non-regexp use case) , which seems to be doing pretty much the same thing...

https://github.com/IAMconsortium/pyam/blob/a4c6c71a20666ac70a1812f77b61d80e18cfd96e/pyam/utils.py#L492

danielhuppmann avatar Jun 01 '21 09:06 danielhuppmann

This is used as a trial for filtering the index-query to the IIASA Database API, see #697

danielhuppmann avatar Aug 31 '22 13:08 danielhuppmann

I reverted the previous PR in #699, because fnmatch does not natively support filtering lists, using the pyam function pattern_match() directly was an easier fix. fnmatch might still be a useful option within pattern_match().

danielhuppmann avatar Sep 09 '22 05:09 danielhuppmann

Looked at this again as a possible improvement to #849, but this package also has issues with square brackets and would require some customization... See https://docs.python.org/3/library/fnmatch.html

danielhuppmann avatar Apr 18 '24 15:04 danielhuppmann