shapefile-rs icon indicating copy to clipboard operation
shapefile-rs copied to clipboard

why the shapefile read .shp Slower than shapefile(pyshp) of python? It doesn't make

Open keepbeer opened this issue 2 years ago • 12 comments

it don't make any sense

keepbeer avatar Apr 28 '24 07:04 keepbeer

@tmontaigu

keepbeer avatar Apr 28 '24 07:04 keepbeer

What about sharing the code you use ? and how you compile, and the file you try to read

Your issue doesn't make sense

tmontaigu avatar Apr 28 '24 09:04 tmontaigu

@kl402401

tmontaigu avatar Apr 28 '24 09:04 tmontaigu

no info = no bug = no fix

tmontaigu avatar May 04 '24 10:05 tmontaigu

no info = no bug = no fix

sorry ,it maybe not a bug, but i use the pyshp of python to read and iterate through the .shp that it has 96w shp, it just cost 30secs,but the same .shp, use shapefile of rust ,it cost 180secs

keepbeer avatar May 06 '24 09:05 keepbeer

I can't do anything about that unless you give some actual info

  • What is the rust code you wrote ?
  • How do you run/compile it (don't forget the --release flag)
  • Are you able to share the file itself ?

tmontaigu avatar May 06 '24 09:05 tmontaigu

sorry ,I can't share the file ,it's a secret file.

.it's a road.shp of the beijing that one of the city in china. it may vary big, it has 9.6 million shape in one .shp。

.the rust code just like that, i build it with --release flag: use std::time::{Instant}; fn main() { let start = Instant::now(); let mut reader = shapefile::Reader::from_path("D:\road0714out\result1\beijing\road.shp").unwrap();

for result in reader.iter_shapes_and_records() {
    let (shape, record) = result.unwrap();
    println!("Shape: {}, records: ", shape);
    for (name, value) in record {
        println!("\t{}: {:?}, ", name, value);
    }
    println!();
}
let duration = start.elapsed();
let seconds = duration.as_secs();

println!("Execution time in seconds: {}", seconds);

}

keepbeer avatar May 06 '24 10:05 keepbeer

Execution time in seconds: 1114 this time it cost 1114secs, is that the code problem? that maybe a little slow

keepbeer avatar May 06 '24 10:05 keepbeer

Does the file have an index file road.shx that goes along with it ? If so I think I have an idea for the where slowness comes from and know how to fix

tmontaigu avatar May 06 '24 11:05 tmontaigu

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

tmontaigu avatar May 06 '24 17:05 tmontaigu

Does the file have an index file road.shx that goes along with it ? If so I think I have an idea for the where slowness comes from and know how to fix

it has a 'road.shx'.

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

That's really great,can you update the project?

keepbeer avatar May 07 '24 01:05 keepbeer

Fix merged in #36

tmontaigu avatar May 12 '24 16:05 tmontaigu