fusen icon indicating copy to clipboard operation
fusen copied to clipboard

Deal with machine not using UTF8 encoding by default

Open statnmap opened this issue 3 years ago • 1 comments

If the machine does not default to UTF-8 encoding, files outputs may lead to encoding problems.
Is suspect it to be due to readr::read_file() in {parsermd} using default UTF8 encoding when reading the file.
Before opening an issue on {parsermd}, I need to test the two Rmd files included in the zip file below on different Windows machines to see if this solves the problem.

encoding_problems_rmds.zip

Maybe this before the inflate() could work too.

Sys.setlocale(category = "LC_ALL", locale = "Spanish_Spain.1252")
# or on the contrary
Sys.setlocale(category = "LC_ALL", locale = "Spanish_Spain.UTF8")
  • [ ] Maybe first try to write the flat file in a temp file: read with system encoding, write in utf8, run parsermd on the temp file.

statnmap avatar Mar 14 '22 06:03 statnmap

Santiago: I created a fusen from the start based on the flat_full template (here is the github repo: https://github.com/aito123/pulso2) and used the suggestion of the Sys.setlocale() before inflate but still getting encoding errors.

# Keep eval=FALSE to avoid infinite loop in case you hit the knit button
# Execute in the console directly
Sys.setlocale(category = "LC_ALL", locale = "Spanish_Spain.1252")
fusen::inflate(flat_file = "dev/flat_full.Rmd", vignette_name = "Get started")

the Readme correctly handles special characters, but that the vignette does not.

statnmap avatar Mar 28 '22 09:03 statnmap

This issue is listed in the future roadmap. It will be re-opened when necessary. See: https://github.com/ThinkR-open/fusen/wiki/Future-road-map

statnmap avatar Jun 02 '23 16:06 statnmap

Now that 'fusen' uses 'lightparser' instead of 'parsermd', that may be good

statnmap avatar May 02 '24 12:05 statnmap