Cargo clone as a library

[tqwewe]

Can I use cargo clone from my Rust app as a library rather than an executable?

It would be nice in my case, rather than spawning a process to run cargo clone.

[tqwewe]

I just realised i can import it as a library 🤦 sorry

[JanLikar]

Hi, yes you can, but its interface is not really designed for it. Maybe it would make sense to improve it.

Do you want to collaborate on this? It would be best if you gave suggestions, since you're the one who'll be using it.

[tqwewe]

Yep I did use it in the end, but It felt like I had to do a lot of boilerplate for simply cloning a crate given name and version into a directory.

My code ended up being:

// Temporary directory to clone into
let dir = tempdir()?;

// Setup default cargo config
let cargo_config =
    Config::default().map_err(|err| Error::Cargo(err.to_string()))?;

// Had to aquire a package cache lock, otherwise I was getting a panic
let package_cache_lock = cargo_config
    .acquire_package_cache_lock()
    .map_err(|err| Error::Cargo(err.to_string()))?;

// Clone a single package... but sadly `Crate` does not implement Clone for some reason.
// So I had to call Crate::new twice.
cargo_clone::clone_single(
    &Crate::new(name.clone(), Some(version.clone())), // Create the crate here
    dir.path(),
    &CloneOpts::new(
        &[Crate::new(name, Some(version))], // Create the crate here again.. as just one item in an array
        &SourceId::crates_io(&cargo_config)
            .map_err(|err| Error::Cargo(err.to_string()))?,
        Some(dir.path().to_string_lossy().as_ref()), // The CloneOpts takes a string for the path.. maybe it should take a Path or PathBuf?
        false,
    ),
    &cargo_config,
)
.map_err(|err| Error::Cargo(err.to_string()))?;

// I had to drop the package cache lock in this order.
// If I put the `drop(..)` line at the end, then it seemed to corrupt my cargo global lock
// and I had to manually run `rm -rf ~/.cargo/.package-cahce`
drop(package_cache_lock);
cargo_config.release_package_cache_lock();

Ultimately what I wanted to do was just: clone(package_name, package_version, dest_path). Also it would be nice if it was async.

I saw an issue about removing cargo as a dependency of this under https://github.com/JanLikar/cargo-clone/issues/41, and I think it might clean things up a bit.. as a lot of the complexity comes from cargo library having so many options I guess.

Take this comment with a grain of salt, I am just being nit picky.

[JanLikar]

Thanks for your feedback, I agree for the most part. I'll see what I can do.

Btw, what are you using cargo-clone for if that's not a secret?

[tqwewe]

Btw, what are you using cargo-clone for if that's not a secret?

Actually the answer is quite complicated. But simplified, it's for my project, I am cloning down a package from either Github or Crate.io (based on what a user selects from the frontend dashboard), and using rustdoc --output-format json to extract specific items from their selected library.

[JanLikar]

That's cool.

I'll try to remember to ping you once I cut a new version.

[marcoieni]

I am also using this package as a library, this is how I am using it at the moment. I am copy pasting since the repository is still private:

use std::{
    fs,
    path::{Path, PathBuf},
};

use anyhow::{anyhow, Context};
use cargo::core::SourceId;
use cargo_metadata::Package;
use tempfile::tempdir;
use tracing::instrument;

#[instrument]
pub fn download_crates(crates: &[&str]) -> anyhow::Result<Vec<Package>> {
    let config = cargo::Config::default().expect("Unable to get cargo config.");
    let source_id = SourceId::crates_io(&config).expect("Unable to retriece source id.");
    let crates: Vec<cargo_clone::Crate> = crates
        .iter()
        .map(|c| cargo_clone::Crate::new(c.to_string(), None))
        .collect();
    let temp_dir = tempdir()?;
    let directory = temp_dir.as_ref().to_str().expect("invalid path");
    let clone_opts = cargo_clone::CloneOpts::new(&crates, &source_id, Some(directory), false);
    cargo_clone::clone(&clone_opts, &config).context("cannot download remote crates")?;
    let crates = if crates.len() == 1 {
        vec![read_package(directory)?]
    } else {
        let crates = sub_directories(directory)?;
        let crates = crates
            .iter()
            .map(|p| read_package(&p))
            .collect::<Result<Vec<Package>, _>>();
        crates?
    };
    Ok(crates)
}

fn sub_directories(directory: impl AsRef<Path>) -> anyhow::Result<Vec<PathBuf>> {
    let mut directories = vec![];
    for entry in fs::read_dir(directory)? {
        let entry = entry?;
        let path = entry.path();
        if path.is_dir() {
            directories.push(path)
        }
    }
    Ok(directories)
}

fn read_package(directory: impl AsRef<Path>) -> anyhow::Result<Package> {
    let manifest_path = directory.as_ref().join("Cargo.toml");
    let metadata = cargo_metadata::MetadataCommand::new()
        .no_deps()
        .manifest_path(manifest_path)
        .exec()?;
    let package = metadata
        .packages
        .get(0)
        .ok_or_else(|| anyhow!("cannot retrieve package at {:?}", directory.as_ref()))?;
    Ok(package.clone())
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    #[ignore]
    fn rand_crate_is_downloaded() {
        let crate_name = "rand";
        let crates = download_crates(&[crate_name]).unwrap();
        let rand = &crates[0];
        assert_eq!(rand.name, crate_name);
    }
}

[marcoieni]

I just realized this code has a problem: be aware that temp_dir is deleted after the function download_crates is done, so you can't the crate from disk once you downloaded it ^^' one workaround is to wrap Vec<Package> in a struct and store temp_dir in the struct itself as a private field.