Forum Discussion

Rajesh_Swamy's avatar
Rajesh_Swamy
Copper Contributor
Jan 24, 2024

How to reduce downloading time for releasing pipeline

When I do a deployment task on a given deployment server, it downloads all linked artifacts for the entire release. So I'm downloading a ton of stuff I don't need.
For Example, if it taking 10 minutes to complete the task and out of 10 minutes, it takes 8 minutes just to download artifacts.

Is there a way to avoid downloading unnecessary artifacts?

  • Take this:

     

    • 1. Use Artifact Filters
      When linking artifacts to a release pipeline, you can specify artifact filters to download only the required files or folders.
    • In the pipeline, configure the artifact download step to include only the necessary files for the deployment.
    • 2. Enable Pipeline Caching
      Azure DevOps supports pipeline caching, which allows you to reuse files or dependencies from previous runs instead of downloading them again.
    • Add a Cache task to your pipeline and configure it to cache the required files or dependencies:
    - task: Cache@2
      inputs:
        key: 'artifact-cache-key'
        path: '$(Pipeline.Workspace)/artifacts'
    
    • This can significantly reduce the time spent downloading artifacts.

    3. Use Deployment Groups or Self-Hosted Agents

    • If you use self-hosted agents, you can maintain a local cache of artifacts on the deployment server. This avoids downloading the same files repeatedly.
    • Deployment groups can also help manage deployments more efficiently by reusing resources.

    4. Optimize Artifact Creation

    • During the build process, ensure that only the necessary files are included in the artifact. Exclude any redundant or unused files to reduce the artifact size.

    5. Shallow Fetch for Git Repositories

    • If your artifacts are stored in a Git repository, use shallow fetch (fetchDepth: 1) to download only the latest commit instead of the entire repository history.

    6. Parallelize Tasks

    • If possible, parallelize tasks in your pipeline to reduce the overall execution time. For example, download artifacts and perform other setup tasks simultaneously.

Resources