How do I creating a full tree?

[Red-Folder]

So, I want to be able to make multiple deletes in a commit.

From my understanding, I believe I need to:

1. Get the recursive tree for the current commit (via Tree.GetRecursive)
2. Amend that tree, then pass to Tree.Create
3. Create a commit for new Tree sha (via Commit.Create)
4. Attach the new commit to the parent

My problem comes in the Tree.Create.

It requires NewTree object. The recursive tree however is IReadOnlyList<TreeItem>.

Now my first assumption was to simply load the NewTree from the IReadOnlyList<TreeItem> - however the TreeItem and NewTreeItem (within the NewTree) are not of the same object hierarchy.

Is the solution to loop through all of the existing IReadOnlyList<TreeItem> and create a NewTreeItem and add to the NewTree?

Or am I barking up the wrong Tree object?

[Red-Folder]

Ok, a bit of playing later ... I believe I have it. Here is sample code:

// Setup tasks
var credentials = new Octokit.Credentials(_username, _key);
var connection = new Octokit.Connection(new Octokit.ProductHeaderValue("Red-Folder.Playground"))
{
    Credentials = credentials
};
var client = new Octokit.GitHubClient(connection);
var parent = await client.Git.Reference.Get(_username, _repo, "heads/master");
var latestCommit = await client.Git.Commit.Get(_username, _repo, parent.Object.Sha);
var currentTree = await client.Git.Tree.GetRecursive(_username, _repo, latestCommit.Tree.Sha);

// Create a new tree from the current tree
var nt = new NewTree();
currentTree.Tree
            .Where(x => x.Type != TreeType.Tree)
            .Select(x => new NewTreeItem
            {
                Path = x.Path,
                Mode = x.Mode,
                Type = x.Type,
                Sha = x.Sha
            })
            .ToList()
            .ForEach(x => nt.Tree.Add(x));

// Remove a file
var toRemove = nt.Tree.Where(x => x.Path.Equals("2017-01-29/TestFile.txt")).First();
nt.Tree.Remove(toRemove);

// Submit the tree, commit and point master at it
var newTree = await client.Git.Tree.Create(_username, _repo, nt);
var newCommit = new NewCommit($"Testing commit", newTree.Sha, parent.Object.Sha);
var commit = await client.Git.Commit.Create(_username, _repo, newCommit);
await client.Git.Reference.Update(_username, _repo, "heads/master", new ReferenceUpdate(commit.Sha));

[Red-Folder]

I'd appreciate any feedback on if this is an appropriate approach.

Note, that I need to remove all the Tree Type nodes as without this, no change is made.

Assuming this is a reasonable hack - would it be a good idea to add a constructor to NewTree which accepts the current tree? (Happy to do the PR if this makes sense)

[ryangribble]

Hey @Red-Folder I haven't played too much with the Tree API so I was poking around trying to figure out the answer to your question. It doesn't sound "right" that you'd have to strip things from the list though

Looking at the API docs https://developer.github.com/v3/git/trees/#create-a-tree I do see that there is a base_tree field that can be set to the sha of the tree you want to update, and we've implemented this BaseTree in NewTree request model.

So rather than loop through the entire recursive tree and add them to a new tree, perhaps you can just have a new tree with the changes you wanted, and set it's BaseTree to the sha of the current tree, avoiding the need to include all the other things you didn't actually want to change... Did you try doing that at all?

In terms of adding this kind of logic to octokit, currently we haven't provided accelerator/composite methods that make multiple API calls, and are more focused on providing equivalence to the underlying API methods. So I'd say it's better to have a documented example of "how to perform complex tasks with octokit" but probably shy away from directly implementing such things. Happy to discuss though

[Red-Folder]

Hi @ryangribble - thanks for the comments

You are 100% correct - there is a base tree. This works great for additions (and I suspect update, but not tried).

It doesn't however support delete.

From the research I've done, I believe that to delete - the only method is to clone the entire existing tree (without setting the base tree). And then either omit or remove the files you wish to delete from the NewTree before submitting it.

If you see the comment in the API docs for the base_tree property - while not explicit in about how it should be used for deleting items, it does point in that direction. And a few blog posts have backed that up.

So yes, I would agree this seems a rather cumbersome approach - I believe it is the appropriate approach. (although always happy to be proved wrong)

[ryangribble]

Ah ok I didn't appreciate the fact that there's no way to delete something except to omit it from a whole new tree !

What's with needing to exclude anything of type Tree from the new tree though? Did you figure out why that's necessary?

[Red-Folder]

Removing the Tree types was taken from this blog post (see section 5b) -> http://www.levibotelho.com/development/commit-a-file-with-the-github-api

Originally I had the above code without the removal of the Tree type - everything seemed to work (I got 201 Created rather than 500 as mentioned in Levi's post). However, when I looked at the commit on Github it showed as no changes had been made.

Add that single line:

.Where(x => x.Type != TreeType.Tree)

And everything seems to be working.

I suspect that having the Tree type in there is the equivalent of setting the base tree (I'd expect them to be same Tree ... not checked that though). As such including the "last" Tree as an object in the list, I believe git is using that as it's base, then applying any additions or updates in the list (of which there are none). The absence of a file in the list has no affect, as the removed file will have already been pulled in when it took the "last" tree.

That last part is speculation on my part ;)

[dkrgpisay]

Hi! Having the same problem as well so I used this as a workaround for now. Are there any plans to make it possible for the deletions to be done via git trees though? I tried to do it based on what the docs mentioned

The SHA1 checksum ID of the object in the tree. Also called tree.sha. If the value is null then the file will be deleted.

but the API always returns an error saying the tree.sha / tree.content should be provided. Hope you can let me know thanks!

[chap-dr]

Holy Smokes, This was a hard google hit to come by, but a lifesaver when found.

Been struggling with bulk removal for a day and a half. I suspect I hit the .Where(x => x.Type != TreeType.Tree) snag, but i replaced my entire messy code bit with the example shown here and it works like a charm.

Only thing i had to change was Type = x.Type to Type = x.Type.Value as i got a Cannot convert source type 'Octokit.StringEnum<Octokit.TreeType>' to target type 'Octokit.TreeType'

The error i was mostly hit by during my stumbling into a solution that worked was: Octokit.ApiValidationException: Must supply tree.sha or tree.content

I'm referencing it here so that this issue may get indexed so it appears in search for that error. :)

[ElPrincidente]

Leaving a side note on this discussion which was very helpful when I encountered a very similar issue.

GitHub API also supports a "partial" recreation of the Tree. This can be very useful when you have a repo which a lot of binary files and/or a high number of TreeItem objects. In my case, the following call was timing out, even if adjusting the GitHubClient timeout via the SetRequestTimeout method (GitHub server does seem to have a 30 seconds hard limit for API calls) var newTree = await client.Git.Tree.Create(_username, _repo, nt); So let's say you only want to delete files in folder A, you can do the following:

• Use client.Git.Tree.Get (non recursive) to retrieve the root tree object
• Copy everything in a NewTree object using the code from above but keep the TreeType.Tree (removing .Where(x => x.Type != TreeType.Tree))
• Use client.Git.Tree.GetRecursive to find tree of Folder A (which we assume it's in the Root Tree)
• Create NewTree object of Folder A tree (then you can then remove the files you want)
• Use client.Git.Tree.Create to commit Folder A New Tree
• Substitute in Root Tree the SHA of Folder A with the TreeResponse received in previous step
• Use client.Git.Tree.Create to commit Root Tree
• Then proceed with the commit and update reference.

This implementation of more complicated but way faster since you are not recreating the full tree, only the elements that requires modifications.

[sPITf1re-dev]

Hey @Red-Folder I haven't played too much with the Tree API so I was poking around trying to figure out the answer to your question. It doesn't sound "right" that you'd have to strip things from the list though

Looking at the API docs https://developer.github.com/v3/git/trees/#create-a-tree I do see that there is a base_tree field that can be set to the sha of the tree you want to update, and we've implemented this BaseTree in NewTree request model.

So rather than loop through the entire recursive tree and add them to a new tree, perhaps you can just have a new tree with the changes you wanted, and set it's BaseTree to the sha of the current tree, avoiding the need to include all the other things you didn't actually want to change... Did you try doing that at all?

In terms of adding this kind of logic to octokit, currently we haven't provided accelerator/composite methods that make multiple API calls, and are more focused on providing equivalence to the underlying API methods. So I'd say it's better to have a documented example of "how to perform complex tasks with octokit" but probably shy away from directly implementing such things. Happy to discuss though

Just saved my day! Trees documentation is si unclear an this part... It says: If not provided, GitHub will create a new Git tree object from only the entries defined in the tree parameter. If you create a new commit pointing to such a tree, then all files which were a part of the parent commit's tree and were not defined in the tree parameter will be listed as deleted by the new commit.

So I tried to omit "base_tree" when creating a tree and tried to use a tree just without the files I want to remove. And it didn't work though the documentation explicitly says that it will be listed as deleted. Just tried to use "base_tree" and commit a tree that contains only files I need to delete (sha is set to null for them) and it finally worked!

[github-actions[bot]]

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!