Don't convert param buffer to utf-8

[roelandxyz]

When a buffer is used as parameter it would always be converted to a utf-8 string. To allow working with a DB that has charset NONE, the buffer is now send directly without converting to string. This could solve #164

You have to take care of your own encoding.

It could be used like this:

const Firebird = require('node-firebird')
const windows1252 = require('windows-1252');

function encodeString(s) {
    return Buffer.from(windows1252.encode(s), 'binary')
}

function decodeString(b) {
    return windows1252.decode(b.toString('binary'))
}

Firebird.attach(options, (err, db) => {
    if (err) throw err;
 
    db.query('INSERT INTO TABLE (NR, FOO) VALUES(?, ?)', 
             [1, encodeString('19 €')], (err, result) => {
        db.query('SELECT * FROM TABLE WHERE NR=?', [1], (err, result) => {
            console.log(decodeString(result[0].FOO))	
            db.detach();
        });
    });
});

[e-baron]

Hello, Thank you very much for this update !!! I have quickly tested it and it worked well. I am very happy as I won't need to use ODBC to work with a DB with NONE charset.

[joaodforce]

I wanted to merge this into my Fork, but the OP deleted his Repo fork :/

[roelandxyz]

Sorry about that. I cleaned up too much (PR is a year old). But I think you can still merge a PR without the fork?

[joaodforce]

The Owner can merge this PR on his master, but I tried to create a PR on my Fork but I could not find your branch do cherrypick a Commit.

I tried adding the original REPO as a second remote to my fork, and fetched it, but your commit doesn't appear. so It seems the PR's commit exists outside the main Repo.

I'm trying to merge the changes by hand lol.

[roelandxyz]

I can make a new PR for this if you like?

[joaodforce]

Its ok, I already applied the patch manually to my Branch. as well as other things.

I discovered this nice tool from GitHub.

https://patch-diff.githubusercontent.com/raw/hgourvest/node-firebird/pull/165.patch

where it shows the patch diff of a particular PR. Sadly I found it after I already copied the code over by hand hahaha.

[pcandido]

@hgourvest @mariuz what do you think about this PR? I am facing the same issue and I would like to collaborate. According to the past messages, I understood @r03 has abandoned this PR, but I can assume and solve the conflict. But my question is why it was not merged before (it is open since 2018). Depending on what is the problem I can contribute in order to approve this PR.

[leonetosoft]

Perfect tank's!

[RonneiPeterson]

I have the same problem, i need put in BD information with acute accent and my field is with charset NONE and collate NONE, but the information save wrong example: café -> Café

[joaodforce]

I have the same problem, i need put in BD information with acute accent and my field is with charset NONE and collate NONE, but the information save wrong example: café -> Café

you have to be mindful of the charset of the existing data on the DB and the data you are trying to put in.

when the charset is NONE, basically what you put it what you read. The main issue is that this Lib applied UTF8 to everything by default. which would be fine if it was the only thing touching the database.

But if you are mixing access with a Windows Program using WIN-1256 Encoding then we have issues, because the bytes are no encoded in the same way. so if you insert "João" using node, then the windows app will decode UTF-8 as WIN-1256 which will result in "Joéo".

And thats the purpose of this PR, which allows Data to be written as raw Bytes to the Fields, so You can encode the string with WIN-1256 using iconv-lite for example. then the text will be correct everywhere.

My fork of this Repo has this PR applied, altho it is behind several commits, it is still used by my company.

[gsimdevelop]

Why this PR was never merged? I have this problem and need it.

[RonneiPeterson]

I resolved this problem setting my table fields to charset WIN1252 and in node-firebird i used with out function that convert in buffer and resolved my problem with charset.

Em seg., 19 de jul. de 2021 às 14:18, João Eduardo @.***> escreveu:

I have the same problem, i need put in BD information with acute accent and my field is with charset NONE and collate NONE, but the information save wrong example: café -> Café

you have to be mindful of the charset of the existing data on the DB and the data you are trying to put in.

when the charset is NONE, basically what you put it what you read. The main issue is that this Lib applied UTF8 to everything by default. which would be fine if it was the only thing touching the database.

But if you are mixing access with a Windows Program using WIN-1256 Encoding then we have issues, because the bytes are no encoded in the same way. so if you insert "João" using node, then the windows app will decode UTF-8 as WIN-1256 which will result in "Joéo".

And thats the purpose of this PR, which allows Data to be written as raw Bytes to the Fields, so You can encode the string with WIN-1256 using iconv-lite for example. then the text will be correct everywhere.

My fork of this Repo has this PR applied, altho it is behind several commits, it is still used by my company.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/pull/165#issuecomment-882759001, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPO43YGQ4FTHJ25JAMQVCDTYRUAPANCNFSM4F7YCRAA .

[joaodforce]

I resolved this problem setting my table fields to charset WIN1252 and in node-firebird i used with out function that convert in buffer and resolved my problem with charset. Em seg., 19 de jul. de 2021 às 14:18, João Eduardo @.***> escreveu: …

Only works if you can make this change, Firebird 2.1 isn't a big fan of Charsets. The library must be agnostic to avoid problems globally.

[gsimdevelop]

I can't change the charset of the tables because is a old BD with thousends of procedures, tables, etc. and all the varchars are defined to NONE charset. Therefore, I haven't another solution. I tried implement this changes in the actually branch but it not work.

[oscarhn27]

Hi, I also need fix this problem. I can't change de charset of my table's varchars either.

[oscarhn27]

Hi, there are any solutions for this problem or somebody developing in it? Thank you.

[RonneiPeterson]

I converted my fields varchar to charset Win1252 and resolved the problem.

Em qua., 1 de set. de 2021 às 02:38, Óscar Hernández Navarro < @.***> escreveu:

Hi, there are any solutions for this problem or somebody developing in it? Thank you.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/pull/165#issuecomment-909955662, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPO4352KTVVRYO7N2T5VLLT7XC5JANCNFSM4F7YCRAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[oscarhn27]

I know that is a solution but I said that can not change this fields charset because is a very big and old database. Thus this pull request is very helpful.

[mreis1]

Guys, why is this PR still on hold? Does this PR introduce a breaking change? or something that needs to be handled?