yaml-cpp
yaml-cpp copied to clipboard
jbeder
0.5.1 Copy construct a YAML::Node lead to memory leak.
I found the boost::shared_ptr are not reset until the program exit.
When I look deeper, I found when I copy construct a YAML::Node from a previous Node, it will leak.
@jbeder It blocks our product release. Could you please take some time help to look at it?
@yyh-graphsql
Do you have some more info on where in the code you see the leak ?
@jkhoogland
when I copy construct a YAML::Node from a previous Node, it will leak.
I think it can be reproduced just by create a YAML::Node and copy it. The leak will be shown when you have many copy operation even you only have one YAML::Node object.
I understood that it occurred when you copied a Node, however the question was did you had any further luck in nailing down where the actual leak occurred in the code... the code is a bit opaque.
I haven't gone further after that. It is difficult for me to dive into the code. Do you have the interest on this issue? Need some help.
yes, i am worried about this, as it impacts my stack as well. As said, the code is difficult to parse. I am willing to spend some time digging in, but it would be nice if the maintainers of this project are also able/willing to provide some input on what might be going on.
The general speed of things moving in this project is pretty low. I see many issues with solutions being PR-ed and just left hanging forever.
@yyh-graphsql could you give a minimal example? Also, make sure you're at HEAD, since the code no longer uses boost.
Hi, Jesse,
We are using version 0.5.1 the version the leak is on. I did try the latest release but it seems there was some incompatibility that caused the yaml config not to work. But still the minimal example should be simple, so I can give you later.
Thanks, Yihao
在 2017年7月11日，17:02，Jesse Beder [email protected] 写道：
@yyh-graphsql https://github.com/yyh-graphsql could you give a minimal example? Also, make sure you're at HEAD, since the code no longer uses boost.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jbeder/yaml-cpp/issues/502#issuecomment-314605433, or mute the thread https://github.com/notifications/unsubscribe-auth/ARNRlRqXGr_IEbB9coUHAXzjN3vFev76ks5sNAz-gaJpZM4N7zky.
The minimum code snippet lists below.
#include <yaml-cpp/yaml.h>
#include <string>
#include <unistd.h>
int main() {
std::string value = "Key: [{}]";
YAML::Node tmp;
for (int i = 0; i < 1000000; ++i) {
YAML::Node node = YAML::Load(value);
//const YAML::Node& nodeS = node["Key"];
tmp = node["key"][0];
if (i%103 == 0) {
::usleep(100000);
}
}
return 0;
}
You will see the memory keeps going up when running the code.
I don't know if this is same; This simple code will make memory accumulate. That is I cannot update the value; every time I update a node, it will allocate new node, not release the old one?
int i = 1000;
while (i-- > 0)
{
some_node["a"] = i;
}
According to this Chinese Post, This "memory leak" is by design caused by the memory management complexity introduced by the section reference feature in YAML syntax. To perfectly solve this issue, GC is required, which is too much for this library.
Not modifying the values could avoid this issue. Only use YAML CPP and a serializer and deserializer.
If you do need to modify the values, serialize it to a string, delete it, and deserialize back.
Any workaround or PR that fixes this? The issue has been here for ~3 years and still nothing?
The issue reproducible by the code above appears to be caused by nodes' m_pMemory not being cleared by assignment operators. You can clear this by serializing your yaml, calling .reset() on any persistent nodes (which will clear the m_pMemory), and then deserializing to repopulate. Just remember that operator= merges the node pools of the source and target, so this only works if the old tree is completely dropped.
As an example, if you add tmp.reset() into the if with the usleep, memory usage will no longer rise forever.
Not exactly a great solution, but it does work
