Download from Source Control

[jessehouwing]

Download the specified file or folder(s) from source control to the specified location.

This can be especially powerful for Release, allowing the ability to pull scripts or tools from source control without having to make them an artifact.

[peder]

We have old XAML build scripts that prompt the developer to choose which branch to publish. As far as I can tell, the new build system does not provide this functionality without perhaps extensive PowerShell scripting. Do you anticipate that a build script could have a variable like $branchPath, steps to download the project's source, and then normal artifact-producing build steps?

[jessehouwing]

It wouldn't be hard to do I suspect, though I'd need a bit more information on what it is you want to accomplish. Can you describe the steps you're missing and what inputs/actions you'd associate to that step?

These steps would basically rule out gated checkin and continuous integration builds... Would that not be an issue?

[peder]

In a nutshell, we need the ability to change which branch we're building from, just like Git repositories have:

[peder]

As far as CI goes, I haven't seen how teams manage to combine CI+TFVC+Feature Branching without creating new build scripts for each feature branch.

[jessehouwing]

The only way I imagine this could work is to either download additional folders to the workspace (as far as you can speak of a workspace, or offer the ability to create additional workspace mappings into which variables are expanded.

While these might work I expect a lot of issues around people trying to use this in CI/Gated/Shelveset builds and I suspect it will cause lots of issues with future branch policies if they ever make their way to TFVC.

Instead you can already achieve this functionality with a variable that you can set from the Queue window and a workspace with "build clean" turned off. as long as developers don't often queue old changeset versions, that should work just fine.

You could create a very simple task to either fail the build if the branch variable isn't set, or set it to a default value using my VSTS Variable Toolkit extension.

[peder]

Hey @jessehouwing, have you made any progress on downloading from source control? We're trying to build this out ourselves but running into module- and assembly-loading nightmares in our homegrown PowerShell scripts.

[danielstefanescu]

Hi @jessehouwing, Any news regarding the download feature?

[peder]

So we ended up writing a script that we call using the PowerShell Task. It downloads files for a selected branch in a new workspace, undoing any local changes before getting latest. It then sets a build variable, Tenaska.SourcesDirectory, that can be used downstream in any build task. We allow the developer to set the $branchPath when they kick off a build by creating a "Settable at queue time" build variable.

Param (
     $agentHomeDirectory,
     $agentBuildDirectory,
     $collectionUri,
     $oauthToken,
     $sourceVersion,
     $workspaceName,
     $branchPath,
     $deleteWorkspace
)

# Set working variables

Write-Output "Download-from-any-branch working variables:"

Write-Output "agentHomeDirectory: $agentHomeDirectory"
Write-Output "branchPath: $branchPath"

$tfPath = $agentHomeDirectory + "\externals\vstshost\tf.exe"

$branchPathFolderStructure = $branchPath.Replace("$", "").Replace("/", "\")

$agentBuildDirectoryDrive = (Get-Item $agentBuildDirectory).PSDrive.Name + ":"
$localFolder = $agentBuildDirectoryDrive + "\src\" + $workspaceName + $branchPathFolderStructure
$serverFolder = $branchPath

$tfWorkspaceCountArgs = "vc", "workspaces", $workspaceName, "/computer:$ENV:COMPUTERNAME", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfCreateWorkspaceArgs = "vc", "workspace", "/new", "/location:local", "/permission:Public", $workspaceName, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfDeleteWorkspaceArgs = "vc", "workspace", "/delete", "/collection:$collectionUri", $workspaceName, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfMapFolderArgs = "vc", "workfold", "/map", "/workspace:$workspaceName",  $serverFolder, $localFolder, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
#$tfGetLatestArgs = "vc", "get", "/version:$sourceVersion", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
# removed /version flag which was not compatible with multiple branches
$tfUndoAllArgs = "vc", "undo", "/recursive", "*", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfGetLatestArgs = "vc", "get", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
#$tfResetWorkspacesArgs = "vc", "workspaces", "/remove:*", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"

$tfWorkspaceCountArgsString = [string]::Join(" ", $tfWorkspaceCountArgs)
$tfCreateWorkspaceArgsString = [string]::Join(" ", $tfCreateWorkspaceArgs)
$tfMapFolderArgsString = [string]::Join(" ", $tfMapFolderArgs)
$tfGetLatestArgsString = [string]::Join(" ", $tfGetLatestArgs)

Write-Output "tfWorkspaceCountArgs: $tfWorkspaceCountArgsString"
Write-Output "tfCreateWorkspaceArgs: $tfCreateWorkspaceArgsString"
Write-Output "tfMapFolderArgs: $tfMapFolderArgsString"
Write-Output "tfGetLatestArgs: $tfGetLatestArgsString"


# Now create the folder, map it to TFS, and emit a SourcesDirectory variable

Write-Output "Attempting to create local directory"
New-Item -ItemType Directory -Force -Path $localFolder | Out-Null

Write-Output "Changing working directory"
Set-Location $localFolder

Write-Output "Determine if workspace exists"
& $tfPath $tfWorkspaceCountArgs
$workspaceMeasure = & $tfPath $tfWorkspaceCountArgs | Measure-Object
$workspaceCount = $workspaceMeasure.Count - 3

Write-Output "Number of workspaces with name $workspaceName`: $workspaceCount"

if($workspaceCount -gt 0) {
    Write-Output "Workspace already exists"

    if($deleteWorkspace -eq $TRUE) {
        Write-Output "Deleting workspace"
        & $tfPath $tfDeleteWorkspaceArgs
        Write-Output "Creating workspace"
        & $tfPath $tfCreateWorkspaceArgs
    }
}
else {
    Write-Output "Creating workspace"
    & $tfPath $tfCreateWorkspaceArgs
}

Write-Output "Mapping folder to workspace"
& $tfPath $tfMapFolderArgs

Write-Output "Undoing any changed files"
& $tfPath $tfUndoAllArgs 2>&1 | Out-Null

Write-Output "Getting latest"
& $tfPath $tfGetLatestArgs

Write-Output "Setting Tenaska.SourcesDirectory variable and changing working location"
Set-Location $localFolder
Write-Output ("##vso[task.setvariable variable=Build.SourcesDirectory;]$localFolder")
Write-Output ("##vso[task.setvariable variable=Tenaska.SourcesDirectory;]$localFolder")

With this in place, we now basically have feature parity with Git Builds. One key missing piece, though: since this is just a bolted-on piece of functionality, the TFS build summary page always shows the same branch, even if a user selected a different $branchPath. I am curious if we change the Build.SourceBranch variable, would TFS save that value back? I'm suspecting we'd actually have to go into the Build table in the SQL database, which obviously would require TFS instead of VSTS. And since this is just to hack in support for multiple branches out of the same build definition for TFVC, there's probably no sense in pushing this any further.

[jessehouwing]

Have you tried setting the source branch variable? That may actually work...

On Dec 19, 2017 13:04, "Peder Rice" [email protected] wrote:

So we ended up writing a script that we call using the PowerShell Task. It downloads files for a selected branch in a new workspace, undoing any local changes before getting latest. It then sets a build variable, Tenaska.SourcesDirectory, that can be used downstream in any build task. We allow the developer to set the $branchPath when they kick off a build by creating a "Settable at queue time" build variable.

 $agentHomeDirectory,
 $agentBuildDirectory,
 $collectionUri,
 $oauthToken,
 $sourceVersion,
 $workspaceName,
 $branchPath,
 $deleteWorkspace

)

Set working variables

Write-Output "Download-from-any-branch working variables:"

Write-Output "agentHomeDirectory: $agentHomeDirectory" Write-Output "branchPath: $branchPath"

$tfPath = $agentHomeDirectory + "\externals\vstshost\tf.exe"

$branchPathFolderStructure = $branchPath.Replace("$", "").Replace("/", "")

$agentBuildDirectoryDrive = (Get-Item $agentBuildDirectory).PSDrive.Name + ":" $localFolder = $agentBuildDirectoryDrive + "\src" + $workspaceName + $branchPathFolderStructure $serverFolder = $branchPath

$tfWorkspaceCountArgs = "vc", "workspaces", $workspaceName, "/computer:$ENV:COMPUTERNAME", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" $tfCreateWorkspaceArgs = "vc", "workspace", "/new", "/location:local", "/permission:Public", $workspaceName, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" $tfDeleteWorkspaceArgs = "vc", "workspace", "/delete", "/collection:$collectionUri", $workspaceName, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" $tfMapFolderArgs = "vc", "workfold", "/map", "/workspace:$workspaceName", $serverFolder, $localFolder, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" #$tfGetLatestArgs = "vc", "get", "/version:$sourceVersion", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"

removed /version flag which was not compatible with multiple branches

$tfUndoAllArgs = "vc", "undo", "/recursive", "", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" $tfGetLatestArgs = "vc", "get", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt" #$tfResetWorkspacesArgs = "vc", "workspaces", "/remove:", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"

$tfWorkspaceCountArgsString = [string]::Join(" ", $tfWorkspaceCountArgs) $tfCreateWorkspaceArgsString = [string]::Join(" ", $tfCreateWorkspaceArgs) $tfMapFolderArgsString = [string]::Join(" ", $tfMapFolderArgs) $tfGetLatestArgsString = [string]::Join(" ", $tfGetLatestArgs)

Write-Output "tfWorkspaceCountArgs: $tfWorkspaceCountArgsString" Write-Output "tfCreateWorkspaceArgs: $tfCreateWorkspaceArgsString" Write-Output "tfMapFolderArgs: $tfMapFolderArgsString" Write-Output "tfGetLatestArgs: $tfGetLatestArgsString"

Now create the folder, map it to TFS, and emit a SourcesDirectory variable

Write-Output "Attempting to create local directory" New-Item -ItemType Directory -Force -Path $localFolder | Out-Null

Write-Output "Changing working directory" Set-Location $localFolder

Write-Output "Determine if workspace exists" & $tfPath $tfWorkspaceCountArgs $workspaceMeasure = & $tfPath $tfWorkspaceCountArgs | Measure-Object $workspaceCount = $workspaceMeasure.Count - 3

Write-Output "Number of workspaces with name $workspaceName`: $workspaceCount"

if($workspaceCount -gt 0) { Write-Output "Workspace already exists"

if($deleteWorkspace -eq $TRUE) {
    Write-Output "Deleting workspace"
    & $tfPath $tfDeleteWorkspaceArgs
    Write-Output "Creating workspace"
    & $tfPath $tfCreateWorkspaceArgs
}

} else { Write-Output "Creating workspace" & $tfPath $tfCreateWorkspaceArgs }

Write-Output "Mapping folder to workspace" & $tfPath $tfMapFolderArgs

Write-Output "Undoing any changed files" & $tfPath $tfUndoAllArgs 2>&1 | Out-Null

Write-Output "Getting latest" & $tfPath $tfGetLatestArgs

Write-Output "Setting Tenaska.SourcesDirectory variable and changing working location" Set-Location $localFolder Write-Output ("##vso[task.setvariable variable=Build.SourcesDirectory;]$localFolder") Write-Output ("##vso[task.setvariable variable=Tenaska.SourcesDirectory;]$localFolder")```

With this in place, we now basically have feature parity with Git Builds. One key missing piece, though: since this is just a bolted-on piece of functionality, the TFS build summary page always shows the same branch, even if a user selected a different $branchPath. I am curious if we change the Build.SourceBranch variable, would TFS save that value back? I'm suspecting we'd actually have to go into the Build table in the SQL database, which obviously would require TFS instead of VSTS. And since this is just to hack in support for multiple branches out of the same build definition for TFVC, there's probably no sense in pushing this any further.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/30#issuecomment-352730556, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uS9uqxJoE1mqXsGZQBMK5BwPT6q9Cks5tB6ZPgaJpZM4ICX-k .

[peder]

I did try that, and I also researched the REST API, which doesn't expose the branch path in their PATCH method. I think the only method available is via UPDATE statements directly against the collection database.

[peder]

https://stackoverflow.com/questions/25858650/update-status-of-tfs-builds-with-powershell

The examples in this post are likely specific to XAML builds, but there may be a way with PowerShell to change data on a Build object, including the Branch Path. I'll take a look at that and get back to this thread in another 1.5 years 👍

[jessehouwing]

Yeah that won't work. And I'm not sure if the rest api will update the value, as some fields are locked to ensure auditability.

On Dec 19, 2017 20:53, "Peder Rice" [email protected] wrote:

https://stackoverflow.com/questions/25858650/update- status-of-tfs-builds-with-powershell

The examples in this post are likely specific to XAML builds, but there may be a way with PowerShell to change data on a Build object, including the Branch Path. I'll take a look at that and get back to this thread in another 1.5 years 👍

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/30#issuecomment-352868153, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uS_BWVWcNNXtGGXcat-UHgnQb52cyks5tCBQigaJpZM4ICX-k .