osrm-extract command with run-time error on Windows OS

[tommybee-dev]

Discussed in https://github.com/Project-OSRM/osrm-backend/discussions/6932

^{Originally posted by tommybee-dev June 8, 2024} Hi, I am new to OSM back-end. Recently I've built this project with MSVC 2022 Express. I've downloaded a pb file frome here.

I've got a runtime error at near by 'next' function since running osrm-extract.exe with the file I downloaded.

osrm-extract data/south-korea-latest.osm.pbf

Here is what I am traced the code from file restriction_graph.cpp.

I don't know how to handle these error. Thanks.

[tommybee-dev]

Hi, Finally, I found the point where the access violation from since I am trying to dig the code in the restriction_graph.cpp.

// Also add any restric
 // tions from suffix paths to the current node in the
 // restriction graph.
 for (const auto &restriction : rg.GetRestrictions(suffix_node))
 {
     insertRestriction(rg, cur_node, restriction);
 }

I changed the original code in the next function above as followed:

// Also add any restric
// tions from suffix paths to the current node in the
// restriction graph.
RestrictionGraph::RestrictionRange range1 = rg.GetRestrictions(suffix_node);

for (int i = 0; i < range1.size(); i++)
{
	const auto &restriction = range1[i];
	insertRestriction(rg, cur_node, restriction);
}

I think the problem is clear that the range has the same memory begin and end. But I still don't know why this problem gonna happened.

Anyone can get me go further will be appreciated.

Thanks.

[mrquangbadao]

Hi Mr @tommybee-dev, I am Quang, your friend here. I applied your solution to the code snippet above and testing it using unit test (using extractor-tests.exe). It’s great news that these block code no longer has errors! However, I noticed another error starting from line 150 in the restriction_graph.cpp file, which had the same root cause as the previous code. I used your solution to fix this code snippet as well, modifying the code From:

for (const auto &suffix_edge : rg.GetEdges(suffix_node)) {
    if (suffix_edge.is_transfer) {
        continue;
    }
//other code

To:

RestrictionGraph::EdgeRange suffix_edges = rg.GetEdges(suffix_node);
for (int i = 0; i < suffix_edges.size(); i++)
{
    const auto &suffix_edge = suffix_edges[i];
    if (suffix_edge.is_transfer)
        continue;
//other code

After making these changes, I re-ran the unit tests, and the error related to the vector loop no longer occurred. And of course, I also tried building and running it again, and the issue has been fixed. osrm-extract.exe ran successfully without any errors.

Thanks.

[donno]

The problem with the following code snippet is the for loop is iterating over the same std::vector that the insert is being performed on. This means the iterators are being invalidated when the vector is resized. In practice, it would get away away with doing what its doing as long as the vector doesn't reallocate (I.e the vector doesn't need to grow and thus allocate a new memory else where), however there does not look like there is any attempt to ensure the reallocation is avoided (i.e no call to std::vector::reserve.

Microsoft's version of the debug runtime for its standard library tracks iterator invalidation so when std::vector::push_back() is called it invalidates the iterators that were previously created/returned and then when the iterator is used (as seen in the original screenshot) it checks if it was invalidated.

            for (const auto &restriction : rg.GetRestrictions(suffix_node))
            {
                insertRestriction(rg, cur_node, restriction);
            }

If the outer loop used index and thus had to look-up the restriction from the vector each time then that would account for if the vector has changed size provided the insertion is always happening at the end (which is what insertRestriction does).

[tommybee-dev]

Thank you for focusing my problem. It is a little bit different story for the problem, because I changed my strategy about solving this problem in which I moved to MinGW environment. Then, I've got a Koean data finally without any problem by modification file - microtar.c and other files. Here is my conclusion. I think It might not be defined the way we use a malloc function on any operating system, windows at this time. All the functions in the files are related to extract the structure from a pbf file. I thought It might not be defined the way we use a malloc function without initialization when to read and extract on any operating system such as Windows at this time, so i'd appended the memset function after malloc in that file. It was main changes and several files. Check my story if you want to know at section 4, '4. 변경한 소스들' at https://tobee.tistory.com/entry/Windows-%ED%99%98%EA%B2%BD-MSYS2-Clang64-%EC%9A%A9-OSRM-%EB%B9%8C%EB%93%9C-%EC%84%A4%EC%B9%98

[jcoupey]

This means the iterators are being invalidated when the vector is resized.

Sounds like this may be a problem on all platforms? In which case we should probably flag this as a bug.

[github-actions[bot]]

This issue seems to be stale. It will be closed in 30 days if no further activity occurs.