Getting started
When running our pipelines in Azure DevOps we are mostly focussed on: will our deployment succeed?, and not so much on any of the regulatory configurations that a pipeline might need to comply to, unless this is something you often have to deal with. One of these things is pipeline retention, the need to keep a pipeline saved for x period of time. This is mostly required for checks to see what has been released to your production environment, but also for very important pipelines in general.
Within Azure DevOps and YAML pipelines in specific there are multiple ways of configuring this retention policy. One of these will be via project settings, which will be applied to all newly ran pipelines after making changes. The other option will be to make a specific HTTP call to Azure DevOps from the YAML pipeline to request an extended retention lease on the pipeline.
NOTE: Any custom retentions via an HTTP call can never be less than the configured days within the settings and never more than 730 days (2 years).
Let’s see how we can apply these two methodes!
Configuring project settings
This is a pretty straightforward way of configuring the basic retention for all pipelines. NOTE: We do need to have the Administrator role on the project in question, to be able to change these settings!
Click on the bottom left corner on Project Settings
And click on the Settings tab underneath the Pipelines category.
As you will see there are multiple settings around retention, such as: Days to keep Artefacts, Runs and more. In our case we want to look at the Days to keep runs which is configured by default to 30 day.
As stated before this can be changed to a maximum of 730 days, but mostly 365 days will suffice for any regulatory purposes. Change this to your needs, if needed.
After changing it (or not), all future pipelines will be retained for the specified amount of days.
But what if you have some pipelines that do need to be retained for more days than your baseline (if not already set to 730 days). Well lets look at some code.
Retaining pipelines via HTTP requests
In my previous blog I showed how to use the REST API Task to make a request to your specified url. This is one of the ways to make such a request to Azure DevOps as well for extending the retention lease to a different amount of days as you configured baseline. Other ways would include making such a request via Powershell or Bash script, so lets have a look at all these different kind of codes options!
REST API Task
trigger:
- none
parameters:
- name: environmentName
type: string
default: prd
pool:
vmImage: ubuntu-latest
stages:
- ${{if eq(parameters.environmentName, 'prd')}}:
- stage:
displayName: Deploy ${{ parameters.environmentName }}
jobs:
- job:
displayName: "Deploy_${{ parameters.environmentName }}"
pool: server
steps:
- task: InvokeRESTAPI@1
displayName: 'Retain on Success'
inputs:
connectionType: connectedServiceName
serviceConnection: DevOps
method: POST
headers: |
{
"Content-type": "application/json",
"Authorization": "bearer $(system.AccessToken)"
}
body: |
[{
"daysValid": 365,
"definitionId": $(System.DefinitionId),
"ownerId": "User:$(Build.RequestedForId)",
"protectPipeline": false,
"runId": $(Build.BuildId)
}]
waitForCompletion: 'false'Powershell
trigger:
- none
parameters:
- name: environmentName
type: string
default: prd
pool:
vmImage: ubuntu-latest
stages:
- ${{if eq(parameters.environmentName, 'prd')}}:
- stage:
displayName: Deploy ${{ parameters.environmentName }}
jobs:
- job:
displayName: "Deploy_${{ parameters.environmentName }}"
steps:
- task: PowerShell@2
condition: and(succeeded(), not(canceled()))
name: RetainOnSuccess
displayName: Retain on Success
inputs:
failOnStderr: true
targetType: 'inline'
script: |
$contentType = "application/json";
$headers = @{ Authorization = 'Bearer $(System.AccessToken)' };
$rawRequest = @{ daysValid = 365; definitionId = $(System.DefinitionId); ownerId = 'User:$(Build.RequestedForId)'; protectPipeline = $false; runId = $(Build.BuildId) };
$request = ConvertTo-Json @($rawRequest);
$uri = "$(System.CollectionUri)$(System.TeamProject)/_apis/build/retention/leases?api-version=6.0-preview.1";
Invoke-RestMethod -uri $uri -method POST -Headers $headers -ContentType $contentType -Body $request;Source Powershell code: Microsoft docs
Bash/ CLI
trigger:
- none
parameters:
- name: environmentName
type: string
default: prd
pool:
vmImage: ubuntu-latest
stages:
- ${{if eq(parameters.environmentName, 'prd')}}:
- stage:
displayName: Deploy ${{ parameters.environmentName }}
jobs:
- job:
displayName: "Deploy_${{ parameters.environmentName }}"
steps:
- task: AzureCLI@2
condition: and(succeeded(), not(canceled()))
name: RetainOnSuccess
displayName: Retain on Success
inputs:
azureSubscription: 'AzureConnection'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
curl -X POST $(System.CollectionUri)$(System.TeamProject)/_apis/build/retention/leases?api-version=6.0-preview.1 -H 'Content-type: application/json' -H 'Authorization: bearer $(system.AccessToken)' -d '[{ daysValid: 365, definitionId: $(System.DefinitionId), ownerId: "User:$(Build.RequestedForId)", protectPipeline: false, runId: $(Build.BuildId) }]' Conclusion
With these 3 options specified, you can choose any that fits your skills, that of your team and/ or company. Allowing you to customise your pipelines retention to any required amount of days (if within 720 days) for any production deployments.
What's next?
With every deployment you make towards Azure, a log of your deployments is being kept. Unfortunately these logs have their limits and you might want to keep them for regulatory purposes. In my next blog we will work on daily exports to keep them nice and safe. Stay tuned!