sparkmagic output html table

[Tagar]

Is there is a way to actually display an html output, like it works without %%spark ?

[Tagar]

@itamarst please have a look if there is a way to fix this ?

thanks!

[winston-zillow]

Same in pyspark kernel, HTML outputs are not displayed but the HTML codes printed. Also the pandas DataFrames (note: not the pyspark DataFrame) are not being rendered prettified, even using display() or display_html().

Whereas running regular kernel (even in the same instance/python env) will render correctly.

[winston-zillow]

@devstein please take a look. This seems an annoying problem.

[devstein]

@winston-zillow Thanks for the example. As demonstrated in first example. Sparkmagic naively renders the response from Livy, which is the { 'text/html': '<h1>H1</h1>' } JSON. This should be fixable. Are you open to contributing?

The second example is much tricky. Your local notebook has access to the raw pandas dataframe while Sparkmagic has to parse the dataframe values from the JSON string returned from Livy. This is what the %%pretty magic attempts to do.