serde-xml-rs icon indicating copy to clipboard operation
serde-xml-rs copied to clipboard

How to ignore attributes/children when deserializing?

Open Boscop opened this issue 8 years ago • 4 comments

How can I parse a XML doc and ignore attributes/members that the xml has, that I don't need in Rust? E.g.:

<?xml version="1.0" encoding="UTF-8"?>
<k foo="asd">
    <pan h="0" v="0" f="90"/>
    <view fisheye="0" limitview="range" fov="75" hlookatmin="-180" hlookatmax="180" vlookatmin="67" vlookatmax="-90" fovmin="40" fovmax="140" hlookat="-20" vlookat="0"/>
    <autorotate enabled="true" waittime="20" speed="1"/>
    <image type="CUBE" multires="false">
        <left url="0_307_2560_50.jpg"/>
        <front url="4_307_2560_85.jpg"/>
        <right url="5_307_2560_50.jpg"/>
        <back url="1_307_2560_50.jpg"/>
        <up url="2_307_2560_85.jpg"/>
        <down url="3_307_2560_40.jpg"/>

        <mobile>
            <left url="0_307_1024_50.jpg"/>
            <front url="4_307_1024_50.jpg"/>
            <right url="5_307_1024_50.jpg"/>
            <back url="1_307_1024_50.jpg"/>
            <up url="2_307_1024_50.jpg"/>
            <down url="3_307_1024_20.jpg"/>
        </mobile>
        <tablet>
            <left url="0_307_1024_50.jpg"/>
            <front url="4_307_1024_50.jpg"/>
            <right url="5_307_1024_50.jpg"/>
            <back url="1_307_1024_50.jpg"/>
            <up url="2_307_1024_50.jpg"/>
            <down url="3_307_1024_30.jpg"/>
        </tablet>
    </image>
</k>

I only care about the <image> so I wrote:

#[derive(Debug, Deserialize)]
struct K {
	#[serde(rename = "$value")]
	items: Vec<KItems>,
}

#[derive(Debug, Deserialize)]
#[allow(non_camel_case_types)]
enum KItems {
	pan(), //{ h: u32, v: u32, f: u32 },
	view,
	autorotate,
	image(Image),
}

#[derive(Debug, Deserialize)]
struct Image {
	#[serde(rename = "type")]
	typ: String,
	multires: bool,
	right: Side,
	left: Side,
	up: Side,
	down: Side,
	front: Side,
	back: Side,
}

#[derive(Debug, Deserialize)]
struct Side {
	url: String
}

But when I try to parse the above doc, the program allocates more and more GB of RAM, even surpassing 18 GB and doesn't return so I have to kill it.

  1. This should never happen. Why does it happen?
  2. How can I ignore all the attributes/child items I don't care about?

I tried pan, pan(), but neither seems to work...

Thanks!

Boscop avatar Jul 17 '17 02:07 Boscop

I think this might be a duplicate of #3 .

This small example you have shown should definitely not require multiple GB of RAM

oli-obk avatar Aug 17 '17 10:08 oli-obk

@Boscop : Hello ! I know I am 3 years late, but I stumbled upon this issue, and it looks like you were trying to parse krpano description files in rust. I am the maintainer of dezoomify-rs, which also contains a (currently incomplete) implementation of the krpano xml format in rust. Can you tell a little bit more on what your project was at the time, and if you happen to still have an open-source implementation of a KRPano XML parser in rust ?

lovasoa avatar Jul 08 '20 12:07 lovasoa

@lovasoa Wow, I didn't know about dezoomify but it looks like a very cool project. That project was about downloading 360° panorama/cubemap tiles and stitching them together, to be able to load them as cubemaps with glium like this. I just looked at the code (that I haven't run since then, because I've been busy with other things, but at some point I might get back to it). I vaguely remember that after I added explicit enum variants for all the cases that can occur, it worked. These are the structs I ended up using, but they are incomplete as well, stuffing all the attributes into Vec<String>s. Feel free to use this & build on this:

#[derive(Debug, Deserialize)]
struct KrPano {
	#[serde(rename = "$value")]
	items: Vec<KrPanoItems>,
}

#[derive(Debug, Deserialize)]
#[allow(non_camel_case_types)]
enum KrPanoItems {
	panoview(Vec<String>),
	view(Vec<String>),
	autorotate(Vec<String>),
	image(Image),
	action(Vec<String>),
	layer(Vec<String>),
}

#[derive(Debug, Deserialize)]
struct Image {
	#[serde(rename = "type")]
	typ: String,
	multires: bool,
	right: Side,
	left: Side,
	up: Side,
	down: Side,
	front: Side,
	back: Side,
}

#[derive(Debug, Deserialize)]
struct Side {
	url: String,
}

Btw, could you add support for downloading tiles from google streetview panoramas? :)

Some example code here: https://github.com/Jam3/extract-streetview https://github.com/Jam3/google-panorama-tiles/blob/master/index.js

E.g.

curl https://lh3.ggpht.com/-g13kt2ekmi8/Vi2AaAydWgI/AAAAAAAAIZY/Md1XU4d_2echRCFXuSf0RQCO4MKDS-FxACJkC/x[0-11]-y[0-5]-z4/p -o tile_#2-#1.jpg
montage -mode concatenate -tile 12x6 tile*.jpg pano.png

But depending on the size, it needs to be cropped:

curl https://lh3.ggpht.com/-4RiMhWapOug/WUHkWYAWSoI/AAAAAAAAcHw/bmNgAVEQCK8JRQtaMRUfVBismGPUbB5_ACLIBGAYYCw/x[0-11]-y[0-5]-z4/p -o tile_#2-#1.jpg
montage -mode concatenate -tile 12x tile*.jpg pano.png
mogrify -crop 5000x2500+0+0 pano.png

Boscop avatar Jul 08 '20 14:07 Boscop

Thank you @Boscop ! For google streetview images, I am not familiar with the format, but it looks like the kind of things that dezoomify-rs's generic dezoomer handles.

lovasoa avatar Jul 08 '20 15:07 lovasoa