phoenix icon indicating copy to clipboard operation
phoenix copied to clipboard

[bug] Phoenix prompt does not preserve original tool definition key order

Open cephalization opened this issue 1 year ago • 3 comments

When users save a prompt with Phoenix, and then fetch that same prompt back, any returned tool definitions have a different key order for their object properties.

While the JSON specification does not mandate the object property order be stable, I do think it is undesirable as a user to save raw json (tool definition) one way, but retrieve it back in another.

cephalization avatar Apr 23 '25 14:04 cephalization

See public discussion on this issue, here https://arize-ai.slack.com/archives/C04R3GXC8HK/p1745390622524729

cephalization avatar Apr 23 '25 14:04 cephalization

Hi,

I've noticed that Phoenix uses the JSONB type in PostgreSQL to store tools in the prompt_versions table. However, JSONB does not preserve the order of object keys—instead, it stores them based on key length, as mentioned here.

To preserve the original key order, we could consider using the JSON type in PostgreSQL instead.

cdaman123 avatar Apr 23 '25 14:04 cdaman123

One thing we can try is to customize the Pydantic serialization and deserialization process specifically to and from the database, so that the function parameters are stored as a (json) string instead.

RogerHYang avatar Apr 23 '25 15:04 RogerHYang

Hello,

When can we expect this to be included in a release?

aman-gupta-doc avatar Apr 28 '25 05:04 aman-gupta-doc

Hey @aman-gupta-doc, this fix will be included in the upcoming v9 release, which you can expect in the coming weeks.

axiomofjoy avatar Apr 28 '25 15:04 axiomofjoy