git2-rs icon indicating copy to clipboard operation
git2-rs copied to clipboard

clone a Repo, The memory has increased from 1m to 20m and will not be released

Open rustbomber opened this issue 4 years ago • 3 comments

I use this code to clone repo, The memory has increased from 1m to >20m and more。And will not be released

fn clone(){

    let mut callbacks = RemoteCallbacks::new();

    callbacks.credentials(|_url, username_from_url, _allowed_types| {
        Cred::userpass_plaintext("abc","abcd)
    });

    // Prepare fetch options.
    let mut fo = git2::FetchOptions::new();
    fo.remote_callbacks(callbacks);

    // Prepare builder.
    let mut builder = git2::build::RepoBuilder::new();
    builder.fetch_options(fo);

    let r= builder.clone(
        "http://xxxx.git",
        Path::new("/code-analyse/"),
    );

    match r{
        Ok(repo) => {println!("ok") }
        Err(e) => {println!("{}",e)}
    }
}

rustbomber avatar Apr 14 '21 04:04 rustbomber

Can you say how you are measuring the memory usage? Allocators sometimes don't release unused memory back to the operating system, so perhaps it was just a spike?

ehuss avatar Apr 14 '21 18:04 ehuss

I just observe the memory situation of the process(code-analyse-server), from the initial <1m, after cloning the repo, it rises to >20m, then it will drop to about 10m, and then keep

just started: WeChatWorkScreenshot_45c62518-72fc-4ee3-aaea-3d0c1c31f0ef

clone some repo: WeChatWorkScreenshot_42e961bc-d470-4b16-aacd-8272f9f06f4b

down to a certain level and keep WeChatWorkScreenshot_5fef478b-ed8c-46c9-a657-478fe728c502

rustbomber avatar Apr 15 '21 02:04 rustbomber

I'm not sure what those screenshots are from, but if that is from a process viewer, I think that would be expected to some degree. As I mentioned, allocators won't always release pages back to the operating system. The memory may be fragmented, or it may be caching pages for quick reuse. It is generally not a concern since the OS can remove unused pages from RAM when needed.

I tried your example with some instrumentation. libgit2_sys::init() allocated about 100k of global stuff because libgit2 has some global structures. Otherwise, I did not see any leaks during the clone operation, and memory was released after the clone call finished.

ehuss avatar Apr 22 '21 00:04 ehuss