Snowfakery icon indicating copy to clipboard operation
Snowfakery copied to clipboard

add encoder for list when using native types

Open darshanime-d11 opened this issue 9 months ago • 4 comments

currently when using native types (when snowfakery_version: 3) is specified, this example recipe from docs fails:

- snowfakery_version: 3
- object: SomeObject
  fields:
    formula_field_example: ${{fake.json(data_columns={'Spec':'@1.0.1',
      'ID':'pyint','Details':{'Name':'name', 'Address':'address'}},
      num_rows=10)}}

this pr adds an encoder for list that generates a valid json that is backwards compatible with version 2

$ cat SomeObject.csv
formula_field_example,float_field,id
"[{""Spec"": ""1.0.1"", ""ID"": 6243, ""Details"": {""Name"": ""Dr. Kyle Jarvis Jr."", ""Address"": ""USNS English\nFPO AE 13563""}}, {""Spec"": ""1.0.1"", ""ID"": 1889, ""Details"": {""Name"": ""Ashley Black DDS"", ""Address"": ""USNV Mcknight\nFPO AA 10708""}}, {""Spec"": ""1.0.1"", ""ID"": 738, ""Details"": {""Name"": ""Isabella Haynes PhD"", ""Address"": ""Unit 1410 Box 9209\nDPO AE 28376""}}, {""Spec"": ""1.0.1"", ""ID"": 2713, ""Details"": {""Name"": ""Lydia Rubio MD"", ""Address"": ""PSC 8156, Box 6235\nAPO AA 56803""}}, {""Spec"": ""1.0.1"", ""ID"": 3728, ""Details"": {""Name"": ""Jay Key II"", ""Address"": ""PSC 3947, Box 8794\nAPO AE 45672""}}, {""Spec"": ""1.0.1"", ""ID"": 2060, ""Details"": {""Name"": ""David Owen"", ""Address"": ""4575 George Drives\nLake Lee, KS 03067""}}, {""Spec"": ""1.0.1"", ""ID"": 9155, ""Details"": {""Name"": ""Dr. Colin Willis"", ""Address"": ""48809 Hull Groves Apt. 854\nSouth Parkerfurt, AK 50804""}}, {""Spec"": ""1.0.1"", ""ID"": 8396, ""Details"": {""Name"": ""Mrs. Nicole Banks"", ""Address"": ""Unit 5502 Box 3002\nDPO AP 52724""}}, {""Spec"": ""1.0.1"", ""ID"": 1926, ""Details"": {""Name"": ""Dr. Fernando Hull"", ""Address"": ""USNS Finley\nFPO AE 91106""}}, {""Spec"": ""1.0.1"", ""ID"": 8764, ""Details"": {""Name"": ""Mr. Anthony Fox Jr."", ""Address"": ""PSC 2680, Box 3112\nAPO AA 36340""}}]",2.2,1

darshanime-d11 avatar Mar 18 '25 13:03 darshanime-d11

Thanks for the contribution! Before we can merge this, we need @darshanime-d11 to sign the Salesforce Inc. Contributor License Agreement.

salesforce-cla[bot] avatar Mar 18 '25 13:03 salesforce-cla[bot]

can you please review this PR @jstvz, @vsbharath

darshanime-d11 avatar Apr 04 '25 11:04 darshanime-d11

i have added the additional types and encoders for them.

sample output:

$ cat /tmp/foo.yaml
- snowfakery_version: 3
- object: foo
  fields:
    a: 1
    c: 3
- object: foo
  fields:
    b: 2
    d: 4
- object: bar
  fields:
    barb: 2
    bard: 4
- object: faz
  fields:
    list_t: ${{fake.json(data_columns={'Spec':'@1.0.1', 'ID':'pyint','Details':{'Name':'name', 'Address':'address'}}, num_rows=2)}}
    dict_t: ${{fake.json(data_columns={'Spec':'@1.0.1', 'ID':'pyint','Details':{'Name':'name', 'Address':'address'}}, num_rows=1)}}
    bool_t: ${{fake.pybool()}}
    tuple_t: ${{(1, 2)}}
    bytes_t: ${{fake.binary(length=64)}}



$ ./venv2/bin/snowfakery /tmp/foo.yaml --output-format csv && cat faz.csv
Created foo.csv
Created bar.csv
Created faz.csv
Created csvw_metadata.json
list_t,dict_t,bool_t,tuple_t,bytes_t,id
"[{""Spec"": ""1.0.1"", ""ID"": 7680, ""Details"": {""Name"": ""Mr. Roberto Gentry"", ""Address"": ""Unit 7523 Box 4996\nDPO AE 41113""}}, {""Spec"": ""1.0.1"", ""ID"": 1399, ""Details"": {""Name"": ""Luke Oconnor"", ""Address"": ""471 Stacey Lakes\nLorimouth, AS 83846""}}]","{""Spec"": ""1.0.1"", ""ID"": 5858, ""Details"": {""Name"": ""Colin Harrington PhD"", ""Address"": ""Unit 6316 Box 3475\nDPO AP 83921""}}",0,"(1, 2)",b'\xbc\xe6\x07\xf3\xdd\xe8p~E~\x1e\x93\x02^r\xbf&\xe0\xd4MB\x95\xa6\xb7\xef:\xf6\xc9\x91\xdb\xa4\x85!CP\x05\xe2\x7f\x965\xc1k\xafU\x11K\xa8\t\x82\\\x9f\x9f\xd7\xaa\x84M\x858\xcev{a\xbbm',1

darshanime-d11 avatar May 05 '25 11:05 darshanime-d11

ptal @jstvz, @vsbharath

darshanime-d11 avatar May 28 '25 14:05 darshanime-d11