Azure DevOps pass variable from Build to Release
Update:
The code is updated to allow for other existing variables in the variable group without removing them.
In my work creating a pipeline for Azure Automation Runbooks I came across a challenge I wanted to share one solution for.
My specific challenge was to only import Runbooks that had changed from one PR to the next.
Should be easy enough right? Just do: git diff –name-only HEAD~…
But wait, I need to do this in the Release part of the pipeline, and this does not have access to my repo out of the box (nor should it need too, you know security and stuff).
So I do know that Build has this out of the box, as it needs to download the bits from github to be able to do the needful. This means I can run git diff there and everything is fine and dandy?
Nope, there is no easy way of passing the output from the command to the Release pipeline. Only interaction point is through the artifacts Build creates for Release to consume, and I do not want to add to that (as it should only be the bits you need to do the deployment) .
Though Azure DevOps has something else called Variable Groups that seems to have been built to handle this type of need. One can share variables created inside the group between Build and Release. Though at this moment there seems not to be a clear way of programmatically set the value of the variable at Build runtime.
Enter the excellent VSTeam module from Donovan Brown that I have had my eye on for some time, but not have had any good excuse of using. Not until now that is.
This module has Get-VSTeamVariableGroup and Update-VSTeamVariableGroup that seems to fit my needs.
So now I seem to have all the bits needed to be able to pass the changed Runbooks from the last PR in to the Release pipeline and have the logic only update these. This is much more efficient than having to import all each time, and saves a lot of time on the agent (as everybody now know time = money in Azure).
Let’s get into the details.
First set up the Variable Group and add the variable the logic will use.
The trick to allow the Build pipeline to change the variable is two folds.
First, on the Variable Group give the Build service account access as shown below (will need to be changed from Read to Administrator).
Next on the Build pipeline agent job add “Allow scripts to access OAuth token”.
The last part is to link the Variable Group to Release. This is done in the variable section as shown below.
Now to make this work from inside the Build pipeline logic, add the MS PS task on the agent and create a script that takes these arguments.
Take special note of a couple of these: System.AccessToken, VSteamAccount and System.TeamProject.
System.AccessToken is the Oauth token we configured the Build pipeline to allow scripts to use. And because it is a secret it needs to be passed in as an argument.
System.TeamProject is another variable provided by the service and holds the name of the DevOps project.
VSteamAccount I have created for convenience.
Make sure this is only the base URL of the AzD instance and do not include the project name in it (https://dev.azure.com/MyInstanceName)
Now for some code.
This is the part that gets the PR file diff and puts it into the GitFileDiff variable in the shared Variable Group.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | <code>Write-Host -Object "Fetching changed files from github since last PR" # Get files changed since last PR $GitResult = git diff --name-only HEAD~ | Where-Object { $_ -like "*/Runbooks/*" } | ForEach-Object { $_.split('/')[-1] } Write-Host -Object "Connecting to Azure Pipeline account: $VSteamAccount in project: $VSteamProject" Set-VSTeamAccount -Account $VSteamAccount -Token $AccessToken -UseBearerToken -ErrorAction Continue -ErrorVariable oErr if ($oErr) { Write-Host -Object "##vso[task.logissue type=error;]Failed to connect to AzD account: $VSteamAccount" Write-Error -Message "Failed to connect to AzD account: $VSteamAccount" -ErrorAction Stop } else { # Get variable group Write-Host -Object "Fetching variable group: $VariableGroupName from project: $VSteamProject" $VariableGroup = Get-VSTeamVariableGroup -ProjectName $VSteamProject -Name $VariableGroupName -ErrorAction Continue -ErrorVariable oErr if ($oErr) { Write-Error -Message "Failed to fetch variable group: $VariableGroupName" -ErrorAction Stop } # Check if object structure has changed for variable group if( $null -ne $VariableGroup.id -and $null -ne $VariableGroup.name ) { # Check if variable groups contains multiple vars already if($VariableGroup.variables) { Write-Host -Object "Variable Group: $($VariableGroup.name) contains variables" if( -not $VariableGroup.variables.PSObject.Properties.Match('ArtifactRootPathName') ) { Write-Host -Object "ArtifactRootPathName does not exist, adding it to variable group: $($VariableGroup.name)" $VariableGroup.variables | Add-Member -MemberType NoteProperty -Name "ArtifactRootPathName" -Value @{ value = $ArtifactRootPathName isSecret = $false } } else { Write-Host -Object "ArtifactRootPathName variable exist, updating value" $VariableGroup.variables.ArtifactRootPathName.value = $ArtifactRootPathName } if($VariableGroup.variables.PSObject.Properties.Match($GitDiffVariableName)) { if($GitResult) { $GitFileDiff = $GitResult -join "," Write-Host -Object "Runbook file changes since last PR:`n$GitFileDiff" # Update variable group with file diff values from github Write-Host -Object "Updating shared variable: $GitDiffVariableName" $VariableGroup.variables.$GitDiffVariableName.value = $GitFileDiff } # Update all Runbooks if force deploy or first time build elseif( $ForceDeploy -or ($env:Version_Major -eq "0" -and $env:Version_Minor -eq "0") ) { Write-Host -Object "Forcing publish of all Runbooks" Write-Host -Object "Updating shared variable: $GitDiffVariableName" $VariableGroup.variables.$GitDiffVariableName.value = "!force!" } else { Write-Host -Object "No Runbooks changed since last PR" Write-Host -Object "Updating shared variable: $GitDiffVariableName" $VariableGroup.variables.$GitDiffVariableName.value = "!nochange!" } } else { if($GitResult) { $GitFileDiff = $GitResult -join "," Write-Host -Object "Runbook file changes since last PR:`n$GitFileDiff" # Update variable group with file diff values from github Write-Host -Object "Adding shared variable: $GitDiffVariableName" $VariableGroup.variables | Add-Member -MemberType NoteProperty -Name $GitDiffVariableName -Value @{ value = $GitFileDiff isSecret = $false } } # Update all Runbooks if force deploy or first time build elseif( $ForceDeploy -or ($env:Version_Major -eq "0" -and $env:Version_Minor -eq "0") ) { Write-Host -Object "Forcing publish of all Runbooks" Write-Host -Object "Adding shared variable: $GitDiffVariableName" $VariableGroup.variables | Add-Member -MemberType NoteProperty -Name $GitDiffVariableName -Value @{ value = "!force!" isSecret = $false } } else { Write-Host -Object "No Runbooks changed since last PR" Write-Host -Object "Adding shared variable: $GitDiffVariableName" $VariableGroup.variables | Add-Member -MemberType NoteProperty -Name $GitDiffVariableName -Value @{ value = "!nochange!" isSecret = $false } } } } $UpdateVariableGroup = ConvertTo-Json -InputObject $VariableGroup -Depth 100 -Compress Write-Host -Object "Updating variable group" $Result = Update-VSTeamVariableGroup -Body $UpdateVariableGroup -ProjectName $VSteamProject -Id $VariableGroup.id -ErrorAction Continue -ErrorVariable oErr if ($oErr) { Write-Error -Message "Failed to update variable group variable: $GitDiffVariableName and ArtifactRootPathName"-ErrorAction Stop } else { Write-Host -Object "Successfully added file diff list from last PR to variable group variable: $GitDiffVariableName and Artifact root path to ArtifactRootPathName" if($Result) { Write-Host -Object $Result } } } else { Write-Error -Message "Shared group variable ID and Name is missing in return object" } }</code> |
Now that the variable has been set by the Build pipeline we will need to consume it in the Release pipeline. This is quite easy after the Variable Group is linked to the pipeline. Just add the variable name as an argument to your PS script that needs to use it.
Easy peasy.
Now puppies…
Happy tinkering!