perspective icon indicating copy to clipboard operation
perspective copied to clipboard

Better string functions in expressions

Open sc1f opened this issue 4 years ago • 3 comments

Feature Request

Description of Problem:

Some common string functions for parsing and creating new strings should be added to the expressions engine using ExprTk:

  • substring(str, start_idx, end_idx) => str[start_idx:end_idx], inclusive of start, exclusive of end (like Python)
  • find(str, regex) => index in str where regex matches
  • match(str, regex) => substring of str where regex matches
  • replace(base_str, regex, replacement) => base_str with replacement at the first match of regex
  • replace(base_str, regex, replacement) => base_str with replacement at all matches of regex
  • contains(str, substr) => true if substr in str

sc1f avatar Sep 01 '21 18:09 sc1f

I'm going to leave some use cases I've found for these in comments, starting with this one.

I have a column in a CSV which is parsed as a string, in the format "${currency} ${currency_symbol}${value}", e.g. "USD $1000". Ideally I'd like 2 columns, Currency and Value, and they should be tolerant of failure to parse. This could be better accomplished with split(), but until there is vector support for Perspective+ExprTK, we could use

// Currency (string)
var idx = find("Bad Column", " ");
if (idx != -1) {
    float(substring("Bad Column", 0, idx))
} else {
    null()
}
// Value (integer)
var idx = find("Bad Column", " ");
if (idx != -1) {
    float(substring("Bad Column", idx, strlen("Bad Column")))
} else {
    null()
}

texodus avatar Sep 07 '21 20:09 texodus

I have longitude/latitude coordinates in the form of a string, e.g. (123, 456), and I'd like to plot it via perspective-viewer-mapbox plugin.

// Longitude
var idx = find("Bad Column", ",");
if (idx != -1) {
    float(substring("Bad Column", 1, idx)))
} else {
    null()
}
// Latitude
var idx = find("Bad Column", ",");
if (idx != -1) {
    float(substring("Bad Column", idx + 1, strlen("Bad Column") - 1))
} else {
    null()
}

texodus avatar Sep 07 '21 20:09 texodus

I have various manually-entered fields with (mis/alt)-spellings I would like to pivot on, for example "NC", "N.C.", "N. C." can be normalized to "NC" with

replace_all("State", '[\.\s]', '')

texodus avatar Sep 07 '21 20:09 texodus