icd icon indicating copy to clipboard operation
icd copied to clipboard

Consider using C pointers to speed up interpretation of string IDs

Open jackwasey opened this issue 6 years ago • 1 comments

Strings are slow. R has an internal 'factor' mechanism, so each unique string only has one memory address. We can exploit this to speed up string processing (assuming same encoding for all strings!)

switch (TYPEOF(id)) {
  // factor or integer IDs
  case INTSXP: {
    idi = as<IntegerVector>(id);
    break;
  }
  case STRSXP: {
    idi = IntegerVector(len);
    // get the global char cache pointers
    // https://cran.r-project.org/doc/manuals/r-release/R-ints.html#The-CHARSXP-cache
    for (R_xlen_t i = 0; i != len; ++i) {
      const char * cstmp = CHAR(STRING_ELT(id, i));
      // push back the memory pointer itself. The global cache may change, but
      // not during this thread's execution?
      // Pointer length may be platform dependent...
      unsigned long pnt = reinterpret_cast<unsigned long>(cstmp);
      idi(i) = pnt;
    }
    break;
  } // switch STRSXP
  case REALSXP: {
    idi = floor((NumericVector)id * 1e6);
    break;
  }
  default:
    stop("ID vector should be numeric, factor or character.");
  } // end switch

jackwasey avatar Jan 01 '19 10:01 jackwasey

R doesn't let us encode a string as UTF-8 if it is just ASCII. While all ICD codes are ASCII we might assume that these strings are therefore unique in the global char cache, and thus we can use the memory pointer for all ICD codes.

jackwasey avatar Feb 14 '19 14:02 jackwasey