multi copybook or not

[geethab123]

I have a copybook which has multiple 01 levels. Can this copy book is parsed as variable length. can I parse like below

	cobolDataframe = spark
    .read
    .format("cobol")
    .option("copybook", v_copybook)
    .option("schema_retention_policy", "collapse_root") //removes the root record headerc
    .option("drop_group_fillers", "false")
    .option("generate_record_id", false) // this adds the file id and record id
    .option("is_record_sequence", "true") // reader to use 4 byte record headers to extract records from a mainframe file
    .option("is_rdw_big_endian", "true")
    //.option("is_rdw_part_of_record_length", false)
    .option("rdw_adjustment", -4)
    .load(v_data)

Please confirm. After parsing I am not getting correct parsed data.

[yruslan]

Yes it can. If your level 1 entries represent segments they should also redefine each other.

01 RECORD1.
...
01 RECORD2 REDEFINES RECORD1.
...

[geethab123]

They are not redefining. Still can be treated as variable length

[AnshumanDwivedi]

Seems Cobrix is not able to process multiple copybooks like syncsort is doing. @CobrixTeam : could you please check and suggest in this regards ?

[yruslan]

Sure, we can take a look. Could you provide an example set of copybooks and a small example data file?

[realraviteja]

@yruslan we are also having similar issue: Quick summary About the issue: In my project i have a requirement to read multiple copybooks as One main Copybook using Cobrix. Example: TRANSACTION.CPY 01 CUSTOMERS 05 PURCHASES FIRST_NAME PIC X (04) LASTNAME_NAME PIC X (04) 01 ORGANIZATION 05 DEPARTMENTS ORG_NAME PIC X (04) VENDOR_NAME PIC X (09) 01 MEDICARE 05 BILLS TREATMENT_TYPE PIC X (04) LOCATION_NAME PIC X (09) 01 MEMBERSHIP 05 PARTNERS PARTNER_TYPE PIC X (04) PARTNER_NAME PIC X (09)

   I have similar kind of layout copybook and i'm using cobrix to parse the data.
   In my research what i understood is i cant read any copy which will have multiple main headers .
   Cause Cobrix can handle only one Main header copybook no multiple.
   
   Please provide your thoughts and resolutions for the above issue.