polynote icon indicating copy to clipboard operation
polynote copied to clipboard

Nested data from Spark Dataset is not displayed in Viz Cell

Open burriad opened this issue 1 year ago • 2 comments

Description When visualizing the contents of a Spark Dataset (even with explicit schema provided in the form of a Java data class) in a Viz Cell, the data in nested fields is not resolved and a placeholder text is displayed.

Reproduction Steps to reproduce the behavior:

  1. Read a dataset from a source (table, file)
  2. Cast it to a defined data class (with nested elements)
  3. Display the loaded contents in a Viz Cell

Expectation The contents of a nested field are visible, preferably with the option to collapse in order to save screen space.

Environment

Otherwise, please provide the following information:

  • Polynote Version:
    • Example: 0.5.1
  • Operating System:
    • Example: Windows 10
  • JVM Version:
    • Example: 17
  • Scala Version:
    • Example: 2.13
  • How you installed Polynote:
    • Custom made Docker file
  • The command you are using to start Polynote:

  • Any other details that might be useful, such as whether you are running Polynote as a service, using a custom script, etc.
    • Polynote is running in an Openshift environment

Configuration If applicable, please attach or paste your Polynote configuration (config.yml)

Notebook If applicable, please attach or paste the notebook file (.ipynb) you were trying to run when you see this bug.

Logs If applicable, please attach or paste the Polynote log

Screenshots If applicable, please attach screenshots to help explain your problem.

Additional context Add any other context about the problem here. polynoteStructProblem

burriad avatar Aug 24 '23 14:08 burriad

This looks like a regression in front-end code... at least, I feel like it used to do something better than [object Object]. Thanks for the report!

jeremyrsmith avatar Aug 24 '23 19:08 jeremyrsmith

I guess some good behavior would be:

  • For an array column, split into multiple rows (with the other columns spanning)
  • For a struct, show something similar to what visualizing a case class looks like (collapsible thing with a truncated one-line representation when collapsed)

jeremyrsmith avatar Aug 24 '23 19:08 jeremyrsmith