Copybook path in jar path

[theonebeyond]

Background [Optional]

I am using spark submit to execute a spark program, and I'm passing the copybook path as a parameter, the path I intent to use is the relative path in my jar package resource. After testing only file system path(either file:/// or hdfs://) are accepted

Question

Does Cobrix support jar path, I want to bring in the copybook packed in my jar. Say /copybook/mybook

[yruslan]

In Cobrix you can't specify the path in the jar. But this is interesting, we can implement it.

In the meantime, you can use a workaround:

import org.apache.commons.io.IOUtils

val copybookContents = IOUtils.toString(getClass.getResourceAsStream("/copybook/mybook"), "UTF-8")

val df = spark.read.format("cobol").option("copybook_contents", copybookContents).load("s3://mybucket/myfile")

[mark-weghorst]

@yruslan I don't know if you have written any code yet for this feature, but here is what I use in my own Cobrix projects:

  def copybookLoader(path: String): String = {
    /*
     Loads copybook from resource bundle and returns the copybook as a string
     */
    val stream: InputStream = getClass.getResourceAsStream(path)
    val lines: Iterator[String] = scala.io.Source.fromInputStream( stream )(Codec.UTF8).getLines
    lines.toList.mkString("\n")
  }

[yruslan]

Thanks @mark-weghorst !

Actually, the code is there(https://github.com/AbsaOSS/cobrix/commit/2f1f615e0c1250bf4683948640a0f93534b339f0). Once the new version of Cobrix is release, you can use jar:// to refer to copybooks inside the JAR.