Peter Johnson

Worth adding that compiling & `make check` succeed on an modern M2 Mac (with an ARM 8.5-A chipset).

This issue should be resolved by https://github.com/openvenues/libpostal/pull/632, please confirm

Digging a little deeper I see that the IDs are stored in an [osmium::index::IdSetDense](https://github.com/osmcode/osmium-tool/blob/eca0e8aef79c64d300aa2f966eb14261e5e62fe1/src/extract/strategy_smart.hpp#L38-L43). The [implementation](https://github.com/osmcode/libosmium/blob/master/include/osmium/index/id_set.hpp#L198) is using a bitmap under the hood, seems I assumed wrong there. I'm not...

I'm not a CPP programmer but it seems that [this line](https://github.com/osmcode/libosmium/blob/master/include/osmium/index/id_set.hpp#L231) is allocating a lot more memory than it really needs. I might be reading this wrong but it seems...

I've generated a custom PBF file which demonstrates the issue: [osmium-oom.pbf](https://s3.amazonaws.com/geocodeearth-missinglink/osmium-oom.pbf) (1.1KB) When I run the following command all the RAM on my system is consumed: ```bash osmium extract -s...

It seems the Bitmask implementation in `osmium::index::IdSetDense` consumes a lot more memory than required, particularly for sparse sets. I wanted to open a ticket to discuss before implementing anything, to...

> ..it is hard to write something that works in all cases, uses a minimum amount of memory, is performant etc. totally agree, this is why I would advocate using...

My CPP is terrible but I managed to butcher together a branch for testing here: https://github.com/osmcode/osmium-tool/compare/master...missinglink:osmium-tool:roaring?expand=1

I've some some basic benchmarking of `master`/`roaring`/`IdSetHybrid` which provides some insights: - the existing `IdSetDense` implementation in master is optimized for speed but not for memory consumption - the `roaring`...

> Could you provide more details there, @missinglink ? GitHub seems to be hiding that info in this comment under an arrow https://github.com/osmcode/osmium-tool/issues/258#issuecomment-1356450059