dataMaid
dataMaid copied to clipboard
Minor suggestion: add option to sanitize atypical unicode
I was working with some survey data in which a number of open-ended text responses included atypical unicode characters that broke latex compilation (though could work with xelatex). These text strings tended to be nonsense input from users so it wasn't essential to include them in the codebook but I found it hard to find the right code to strip them out before running dataReporter. Ultimately, it turned out to be easy with the following code gsub('[^\x20-\x7E]', '', text)
but it took me a while to locate this particular solution on StackOverflow (see link below).
https://stackoverflow.com/questions/38828620/how-to-remove-strange-characters-using-gsub-in-r
I wonder if an option/argument to automatically sanitize character strings would make sense given that this will likely be an issue for a wide range of data sets.
Thanks for reaching out!
We are a bit hesitant about implementing features that alters the data even though it's just for the sake of formatting.
Would you mind providing a bit more info about the encoding problem your were having? What did the text strings look like?
It sounds like you have found a good workaround. But as an alternative, you could also run makeDataReport
with standAlone = FALSE
- this gives you a .rmd document without the YAML header - and then make the YAML header yourself. This would allow you to choose which latex engine to use, include a .tex header and more.