I’m excited to announce the public previews of pipeline caching and pipeline artifacts in Azure Pipelines. Together, these technologies can make every run of your pipeline faster by accelerating the transfer of artifacts between jobs and stages, and by caching the results of common operations like package restores.
Pipeline caching
Pipeline caching introduces a new CacheBeta task that takes a path of files to cache and a cache key. A cache key can be the contents of a file (like a package lockfile), a string of your choice, or a combination of both.
For example, to cache Node.js dependencies installed with Yarn:
steps:
- task: NodeTool@0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js 10.x'
- task: CacheBeta@0
inputs:
key: |
$(Agent.OS)
$(Build.SourcesDirectory)/yarn.lock
path: $(Pipeline.Workspace)/.cache/yarn
displayName: 'Cache yarn'
- script: yarn install
As we’ve implemented pipeline caching, we’ve learned that every tool behaves differently. So, we’re excited to release this preview and see the cache used in the wild. If you try out the cache and find it doesn’t improve the performance of the step you’re caching, we’d like to hear about it as an issue on azure-pipelines-tasks. If you can create a public repo and pipeline that we can fork to reproduce your issue, all the better. We’ll be listening to these issues and tuning the cache.
We’ve already got some improvements, including preserving file attributes and fallback keys, that we’ll be shipping while the preview is running. We look forward to your ideas and feedback.
There’s a possibility we’ll make breaking changes between v0 and v1 of the task, so we recommend not yet including the cache in production/master branch CI builds.
Learn more about pipeline caching on docs.microsoft.com.
Pipeline artifacts
Pipeline artifacts are the built-in way to move files between jobs and stages of your pipeline and to save files from your pipeline for later use. Pipeline artifacts intelligently deduplicate content and only upload net-new content to the server, which helps speed up repeated builds of the same repository.
To use Pipeline artifacts, just use the publish and download YAML shortcuts, like this:
steps:
- publish: bin
artifact: binaries
By default, all artifacts published by previous jobs are downloaded at the beginning of subsequent jobs, so it’s not necessary to add a download step. If you want to control this behavior, use the download shortcut, like this:
steps:
- download: none
For most common use cases, the publish and download shortcuts are recommended. However, if you need more control over how your artifacts are downloaded, you can also use the Download Pipeline Artifact and Publish Pipeline Artifact tasks directly.
Learn more about pipeline artifacts on docs.microsoft.com. And, if you run into issues, let us know on Developer Community.
Is it possible to use this task with designer ?
Hi Alex,
Seems this is interesting feature. We are in process of creating pipeline for a huge project where in a single Azure project we will have around 700+ java azure repo and so build pipeline.
The build process is in maven where we will use this heavily but our concern is if we are making pom.xml as key (we have main and child poms) how this solution will benefit. As for every branch there will be different cache(what i understood from documents). How can we achieve cache on Azure project level apart from having cache on each pipeline and...
There is a different type of transportation, and my topic is Auto transport which is the process of shipping cars, please visit our site for more detail Auto shipping services in usa.
Hi Alex,
Thanks for this awesome news. I experimented with this and I must say I stopped using it and reverted back to the pipeline cache solution from MS Devlabs: https://marketplace.visualstudio.com/items?itemName=1ESLighthouseEng.PipelineArtifactCaching
The main reason is that the latter is much faster in restoring caches, see my screenshots for a comparison.
Ok I apparently am unable to insert screenshots in this forum, if you contact me I am happy to provide them for you.
I tested with a node_modules size of 434.8MBBeta caching: 1m 10sMS Devlabs solution: 23s
I felt like sharing, so I wrote a blogpost about this. You can find it here: https://xebia.com/blog/caching-your-node-modules-in-azure-devops/