pg2
pg2 copied to clipboard
igrishaev
Not able to use custom OIDs for parameters
Could you please share what OID you're trying to use? Maybe the the quick fix will be to add it to the enum.
I'm actually making use of the postgres port of XTDB. XTDB introduces a custom OID for specifying the Transit type for a parameter. I believe there are also other DBs out there that provide compatibility with the Postgres wire protocol.
Meanwhile, I'll try to reproduce it locally.
(OID/ofInt 16384) => DEFAULT oid
I'm in the middle of refactoring, and, in general, it works. Now OIDs are not instances of Enum but plain integers.
But I also need to decide about extendable encoding and decoding. Say, how to decode the value from bytes when the OID is non-standard. And how to encode it to bytes when passing into the database.
By any chance, could you please point me to the source code where XTDB processes that Transient type?
I know that XTDB is accepting a pgwire-text-encoded parameter of type transit+json.
It should be a responsibility of the user of the pg2 library to encode the parameter properly. It could either admit the readily serialized parameter, or set up a serialize function for that OID.
By the way: did you pass the OID/DEFAULT oid value? If the parameter is string, and the OID is DEFAULT, it will be passed as-is with no coercion.
Yes, I did. OID/DEFAULT won't work in my case, as XTDB is schemaless and needs client type hints for data being INSERTed.
(Thanks for looking into this btw. If you'd need some involvement from my side, let me know).
Sorry for delay, I've been interrupted at work. Could you please give me an example of how the Transient type is declared in Postgres? I mean, a SQL expression. Is a tuple of primitives, or just an alias to another type?
Hi Ivan. I don't think we have a SQL representation of Transit. It is only a type for serialization of query parameters.
Regarding on how to adapt this library, it'd be enough to accept numbers for OIDs, so not necessarily picked from an enum.
I've pushed a snapshot version called 0.1.18-SNAPSHOT. In this version, oids are not enums any longer but plain integers. It means you can pass any oid you want.
There is no need to specify the oid in fact. If the OID is unknown, the client passes a string value to the server as is with no encoding. Thus, just pass your Transit data as a string, and it must work.
Let me know please if it works.
Meanwhile, I'm working on other improvements that will make type system really flexible.
@egg-juxt could you please give it a try with the latest 0.1.18 release? Or point me to your work (a repo maybe) so I could contribute?
Thanks @igrishaev, noted :pray:
We'll take more of a look next week, but I suspect these are the tests that we'll be able to update/augment with numeric OIDs (Ernesto will be able to confirm).
Cheers,
James
@egg-juxt @jarohen I took a look at these tests, and I'm not sure if I have full context.
As far as I understand, you're implementing Postgres Wire protocol in XTDB, and you would like to test if it works with JDBC. Am I right?
And you have introduced PG2 to check how does XTDB react on wrong OIDs passed, right?
As far as I understand, you're implementing Postgres Wire protocol in XTDB, and you would like to test if it works with JDBC. Am I right?
Yep :slightly_smiling_face:
And you have introduced PG2 to check how does XTDB react on wrong OIDs passed, right?
That's one aspect, yep - but mainly that we'd just like users to be able to have the choice of using PG2 to talk to XT, so we're writing a test suite using it :slightly_smiling_face:
It's not necessarily wrong OIDs, either - we have a couple of extension types that we'd like to be able to round-trip (e.g. Transit)
I suspect this change unblocks us, will poke @egg-juxt to confirm.
Thanks!
Hi Ivan. Yes, my main goal was to add some test into XTDB that exercised our ability to accept Transit objects, so a custom OID, not necessarily an incorrect OID. (Some more context: I was having an issue when connecting from JavaScript using PostgresJS, and I wanted to check using Clojure directly). I'll be back next Monday and I will try it out.
@igrishaev , could you have a look at https://github.com/xtdb/xtdb/pull/3857?
Plain OID numbers must be passed as an Integer, :oids does not accept Long, which is the Clojure default for integer literals.
Looking... yes that int/long coercion between Java and Clojure is always pain.
Ah, I know why didn't I face this issue in my tests, although I have some cases where I pass wrong OIDs. The reason is, I use values from the pg.oid namespace where all the values declared as 4-byte integers. But when you pass a pure number like 1, it's Long indeed. Let me think how to figure out.
For now, you can either replace plain numbers with vars from the pg.oid namespace, or coerce them with (int) while I'm fixing this.