node-firebird icon indicating copy to clipboard operation
node-firebird copied to clipboard

Strings are converted to the data type of buffer

Open khusamov opened this issue 8 years ago • 16 comments

Hello! I have the following table:

CREATE TABLE GROUP_ORDER (
    ID                 INTEGER NOT NULL,
    SOURCE_ID          INTEGER NOT NULL,
    NUMBER             VARCHAR(200) NOT NULL,
    IS_OPTIMIZED       INTEGER NOT NULL,
    "DATE"             TIMESTAMP NOT NULL,
    NUMBER_ORDER_LIST  VARCHAR(5000)
);

I make this request:

select * from GROUP_ORDER

And I get this answer:

[ { ID: 2,
    SOURCE_ID: 5,
    NUMBER: <Buffer 31>,
    IS_OPTIMIZED: 0,
    DATE: 2017-01-16T22:00:00.000Z,
    NUMBER_ORDER_LIST: <Buffer 30 32 36 33 31 5f 30 32 36 31 34> },
  { ID: 534,
    SOURCE_ID: 3,
    NUMBER: <Buffer 33>,
    IS_OPTIMIZED: 1,
    DATE: 2016-04-19T22:00:00.000Z,
    NUMBER_ORDER_LIST: <Buffer 31 36> },
  { ID: 539,
    SOURCE_ID: 9,
    NUMBER: <Buffer 42 49 47 5f 31>,
    IS_OPTIMIZED: 1,
    DATE: 2016-05-21T22:00:00.000Z,
    NUMBER_ORDER_LIST: <Buffer 31 2c 20 32 2c 20 33> },
  { ID: 540,
    SOURCE_ID: 10,
    NUMBER: <Buffer 42 49 47 5f 32>,
    IS_OPTIMIZED: 1,
    DATE: 2016-05-21T22:00:00.000Z,
    NUMBER_ORDER_LIST: <Buffer 31 32 33 2c 20 35 2c 20 36> } ]

Why, instead Strings Variables I have variables of type Buffer (see fields NUMBER and NUMBER_ORDER_LIST)?

On database this one in the form of a buffer to another as a string.

khusamov avatar Jan 30 '17 15:01 khusamov

Hello, I guess it's firebird that converts varchar(5000) to blob of type text. Try setting to less than 5000, for example 500 and you'll get strings in return.

On Jan 30, 2017 16:14, "Khusamov Sukhrob" [email protected] wrote:

Hello! I have the following table:

CREATE TABLE GROUP_ORDER ( ID INTEGER NOT NULL, SOURCE_ID INTEGER NOT NULL, NUMBER VARCHAR(200) NOT NULL, IS_OPTIMIZED INTEGER NOT NULL, "DATE" TIMESTAMP NOT NULL, NUMBER_ORDER_LIST VARCHAR(5000) );

I make this request:

select * from GROUP_ORDER

And I get this answer:

[ { ID: 2, SOURCE_ID: 5, NUMBER: <Buffer 31>, IS_OPTIMIZED: 0, DATE: 2017-01-16T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 30 32 36 33 31 5f 30 32 36 31 34> }, { ID: 534, SOURCE_ID: 3, NUMBER: <Buffer 33>, IS_OPTIMIZED: 1, DATE: 2016-04-19T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 36> }, { ID: 539, SOURCE_ID: 9, NUMBER: <Buffer 42 49 47 5f 31>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 2c 20 32 2c 20 33> }, { ID: 540, SOURCE_ID: 10, NUMBER: <Buffer 42 49 47 5f 32>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 32 33 2c 20 35 2c 20 36> } ]

Why, instead Strings Variables I have variables of type Buffer (see fields NUMBER and NUMBER_ORDER_LIST)?

On database this one in the form of a buffer to another as a string.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/issues/125, or mute the thread https://github.com/notifications/unsubscribe-auth/ACaj6r2cxyqizEaYmqqD0B_T9U7yNae0ks5rXf5egaJpZM4LxhF- .

slachtar avatar Jan 30 '17 15:01 slachtar

And why on the same database it does and the other does not? And how to recognize where the text is, and where the actual binary data?

khusamov avatar Jan 30 '17 15:01 khusamov

Oh my bad, I haven't noticed the varchar(200). Which node version do you use?

On Jan 30, 2017 16:18, "Slah Lachtar" [email protected] wrote:

Hello, I guess it's firebird that converts varchar(5000) to blob of type text. Try setting to less than 5000, for example 500 and you'll get strings in return.

On Jan 30, 2017 16:14, "Khusamov Sukhrob" [email protected] wrote:

Hello! I have the following table:

CREATE TABLE GROUP_ORDER ( ID INTEGER NOT NULL, SOURCE_ID INTEGER NOT NULL, NUMBER VARCHAR(200) NOT NULL, IS_OPTIMIZED INTEGER NOT NULL, "DATE" TIMESTAMP NOT NULL, NUMBER_ORDER_LIST VARCHAR(5000) );

I make this request:

select * from GROUP_ORDER

And I get this answer:

[ { ID: 2, SOURCE_ID: 5, NUMBER: <Buffer 31>, IS_OPTIMIZED: 0, DATE: 2017-01-16T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 30 32 36 33 31 5f 30 32 36 31 34> }, { ID: 534, SOURCE_ID: 3, NUMBER: <Buffer 33>, IS_OPTIMIZED: 1, DATE: 2016-04-19T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 36> }, { ID: 539, SOURCE_ID: 9, NUMBER: <Buffer 42 49 47 5f 31>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 2c 20 32 2c 20 33> }, { ID: 540, SOURCE_ID: 10, NUMBER: <Buffer 42 49 47 5f 32>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 32 33 2c 20 35 2c 20 36> } ]

Why, instead Strings Variables I have variables of type Buffer (see fields NUMBER and NUMBER_ORDER_LIST)?

On database this one in the form of a buffer to another as a string.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/issues/125, or mute the thread https://github.com/notifications/unsubscribe-auth/ACaj6r2cxyqizEaYmqqD0B_T9U7yNae0ks5rXf5egaJpZM4LxhF- .

slachtar avatar Jan 30 '17 15:01 slachtar

Node.js is version 6.9.4

khusamov avatar Jan 30 '17 15:01 khusamov

I'm using the same version of nodewith no problem. BTW I'm using fb version 2.5. As a workaround you may try to cast those problematic fields to varchar in your query.

On Jan 30, 2017 16:25, "Khusamov Sukhrob" [email protected] wrote:

Node.js is version 6.9.4

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/issues/125#issuecomment-276091538, or mute the thread https://github.com/notifications/unsubscribe-auth/ACaj6n8XNiNV6WtCY9gIceFwWb_ZTVrSks5rXgDggaJpZM4LxhF- .

slachtar avatar Jan 30 '17 15:01 slachtar

I firebird version 2.5.4 But I have one computer is working correctly, and on the other, instead of strings returned to the buffer.

khusamov avatar Jan 30 '17 15:01 khusamov

Hi,

You probably missed to define a default character set when creating the database, I suggest using utf8.

On 30/01/2017 16:14, Khusamov Sukhrob wrote:

Hello! I have the following table:

|CREATE TABLE GROUP_ORDER ( ID INTEGER NOT NULL, SOURCE_ID INTEGER NOT NULL, NUMBER VARCHAR(200) NOT NULL, IS_OPTIMIZED INTEGER NOT NULL, "DATE" TIMESTAMP NOT NULL, NUMBER_ORDER_LIST VARCHAR(5000) ); |

I make this request:

|select * from GROUP_ORDER|

And I get this answer:

|[ { ID: 2, SOURCE_ID: 5, NUMBER: <Buffer 31>, IS_OPTIMIZED: 0, DATE: 2017-01-16T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 30 32 36 33 31 5f 30 32 36 31 34> }, { ID: 534, SOURCE_ID: 3, NUMBER: <Buffer 33>, IS_OPTIMIZED: 1, DATE: 2016-04-19T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 36> }, { ID: 539, SOURCE_ID: 9, NUMBER: <Buffer 42 49 47 5f 31>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 2c 20 32 2c 20 33> }, { ID: 540, SOURCE_ID: 10, NUMBER: <Buffer 42 49 47 5f 32>, IS_OPTIMIZED: 1, DATE: 2016-05-21T22:00:00.000Z, NUMBER_ORDER_LIST: <Buffer 31 32 33 2c 20 35 2c 20 36> } ] |

Why, instead Strings Variables I have variables of type Buffer (see fields NUMBER and NUMBER_ORDER_LIST)?

On database this one in the form of a buffer to another as a string.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hgourvest/node-firebird/issues/125, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlTBLCRgguYqnAffxVSC22-6527yxdWks5rXf5egaJpZM4LxhF-.

hgourvest avatar Jan 30 '17 16:01 hgourvest

I utf8 database image

khusamov avatar Jan 30 '17 16:01 khusamov

Hello! There are ideas? Or it is necessary to debug the source code drivers?

khusamov avatar Feb 04 '17 16:02 khusamov

+1 Same problem here! Using Firebird 2.1.

nunesbeto avatar Feb 10 '17 12:02 nunesbeto

You can always cast Buffer buff.toString('utf8')

But is there any way to set connector encoding to win1250, because this encoding is not supported by buffer and I have to use: iconv.decode(buf, 'win1250')

johnymachine avatar May 03 '17 20:05 johnymachine

Same issue here but I want to be able to use JSON.stringify on the result object and don't want to have to go through and convert / case each field in the result to a string. Is there another way? db.query("SELECT first 10 NAME, NUMBER FROM site", function( ... res.send(JSON.stringify(sites));

jamesarm97 avatar Sep 14 '18 20:09 jamesarm97

I do something like this. Not perfect, but it works for now.

    db.execute(sql, (err, data) => {
      if (err) return reject(err);

      for (var i = 0, ilen = data.length; i < ilen; i++) {
        data[i] = data[i].map(field => {
          if (field.buffer) {
            return windows1252.decode(field.toString("binary"));
          }
          return field;
        });
      }
      db.detach();
      resolve({ data });
    });

roelandxyz avatar Sep 15 '18 14:09 roelandxyz

use: select ID, SOURCE_ID, cast(NUMBER as varchar(50) character set utf8) "NUMBER", IS_OPTIMIZED, DATE, cast(NUMBER_ORDER_LIST as varchar(50) character set utf8) "NUMBER_ORDER_LIST" from GROUP_ORDER

cesarbalbinott avatar Sep 25 '18 00:09 cesarbalbinott

Hello, I have used this solution in order to decode buffer objects:

  1. Find out what is the default character set of your DB. I did not know, so I tested those listed here https://nodejs.org/api/buffer.html (utf8, latin1...)
  2. Use function toString( 'your character set') on the buffer objects.

Here is an example of code illustrating how to print to the console the string encoded in the buffer:

db.query('SELECT FIRST 10 NOM,PRENOM FROM ETUD', function(err, result) {
            if (result != undefined)
                { 
                for (i=0; i< result.length ; i++){
                    /* string / text in firebird are converted to buffer objects by NodeJS. Those buffer are encoded in latin1, apparently the default character set of our DB */  
                    console.log("Student: " , result[i].NOM.toString('latin1'), " ", result[i].PRENOM.toString('latin1'));
                }                

                }
            else
                console.log("No results");
            db.detach();
        });

e-baron avatar Oct 10 '18 07:10 e-baron

https://github.com/hgourvest/node-firebird/wiki/Example-of-querying-using-Promises

FabianoCastro avatar Aug 20 '21 04:08 FabianoCastro