odbc icon indicating copy to clipboard operation
odbc copied to clipboard

Encoding/Collation problems with 1.5.0 SQL server

Open soetang opened this issue 6 months ago • 10 comments

We have been expriencing encoding problems with odbc 1.5.0 on SQL server. This works fine on odbc 1.4.2

All our databases unfortunately uses the varchar collation "Danish_Norwegian_CI_AS" - with odbc 1.4.2 however we were able to create a connection so that we could correctly read and write to the database. With odbc 1.5.0 the column names are no longer formatted correctly.

Is this a bug or can we change the connection settings so that it works correctly?

library('magrittr')
conn <- DBI::dbConnect(
  odbc::odbc()
  , Driver = "ODBC Driver 17 for SQL Server"
  , Server = "server"
  , Database = 'database'
  , Trusted_Connection = "Yes"
  , encoding = 'latin1'
  , AutoTranslate = 'no'
)

test_table <- DBI::Id(schema = 'test', table = 'test_table')
df <- tibble::tibble(
  var_char_col = c('kanin', 'ræven', 'ålens', 'ørred'),
  bøvs = 1
)


DBI::dbWriteTable(
  conn
  , name = test_table
  , df
  , field.types = c('var_char_col' = 'varchar(5)')
  , overwrite = TRUE
)

db_data <- DBI::dbReadTable(conn, test_table) %>%
  tibble::as_tibble()

db_data
#> # A tibble: 4 × 2
#>   var_char_col bÃ.vs
#>   <chr>        <dbl>
#> 1 kanin            1
#> 2 ræven            1
#> 3 ålens            1
#> 4 ørred            1

Expected value:

db_data
#> # A tibble: 4 × 2
#>   var_char_col  bøvs
#>   <chr>        <dbl>
#> 1 kanin            1
#> 2 ræven            1
#> 3 ålens            1
#> 4 ørred            1

If we remove the encoding parameter - we get correct colum names, however the character vector no longer fit within the varchar(5) datatype even though we only have 5 characters:

library('magrittr')
conn <- DBI::dbConnect(
  odbc::odbc()
  , Driver = "ODBC Driver 17 for SQL Server"
  , Server = "server"
  , Database = 'database'
  , Trusted_Connection = "Yes"
  # , encoding = 'latin1'
  , AutoTranslate = 'no'
)

test_table <- DBI::Id(schema = 'test', table = 'test_table')
df <- tibble::tibble(
  var_char_col = c('kanin', 'ræven', 'ålens', 'ørred'),
  bøvs = 1
)


DBI::dbWriteTable(
  conn
  , name = test_table
  , df
  , field.types = c('var_char_col' = 'varchar(5)')
  , overwrite = TRUE
)
#> Error in eval(expr, envir, enclos): nanodbc/nanodbc.cpp:1783: 00000
#> [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation

db_data <- DBI::dbReadTable(conn, test_table) %>%
  tibble::as_tibble()

db_data
#> # A tibble: 0 × 2
#> # ℹ 2 variables: var_char_col <chr>, bøvs <dbl>

soetang avatar Aug 23 '24 09:08 soetang