Robust Scripts in Azure Pipelines

Introduction

When developing software, CI/CD pipelines are often used to unit test the code, perform static analysis, deploy artifacts to staging environments, or roll out features to users. Scripts are frequently used to glue programs together.

How Azure Pipelines Scripts Work

Azure Pipelines, part of the Azure DevOps suite of tools, is a cloud-based service that automates the building, testing, and deployment of code projects. It supports continuous integration (CI) and continuous delivery (CD), allowing developers to automate their workflows efficiently. Azure Pipelines can run scripts using different interpreters depending on the platform. For this article, we focus on bash as used for the Linux (Ubuntu) and macOS runners.

A typical Azure Pipeline defined in YAML might look like this:

pool:
  vmImage: 'ubuntu-latest'

steps:
- script: |
    echo "Hello, World!"
    cp ./somefile somewhere
    cat somewhere/somefile | grep things
  displayName: 'Run custom script'

In this example, the script step runs a simple illustrative bash script on an Ubuntu agent. Azure Pipelines supports various properties for scripts, such as setting the working directory, environment variables, and handling errors. This is not dissimilar to other CI/CD environments and providers, such as GitLab CI/CD and GitHub Actions.

Behind the scenes, the script step creates a bash script with the script contents and executes them. Each line in the script is executed individually and in succession, meaning that exit codes are not checked by default.

Even when these script blocks are only used as glue, it can be quite challenging to debug, when a build fails or the pipeline in general fails. I noticed, personally, that errors were not caught early on but silently ignored, only to creep up later in a build and lead to errors. This can be the case, for example, when zipping files and extracting or deploying them later on. What happens if a file is missing, an environment variable is not set?

Unofficial Bash Strict Mode

Using the unofficial bash strict mode in your scripts within Azure Pipelines, or really any script, can significantly enhance the robustness and reliability of your automation processes.

But what is the unofficial bash strict mode?

set -euxo pipefail
IFS=$'\n\t'

Here's a breakdown of what each of these settings does:

1. set -e: This option causes the script to exit immediately if any command exits with a non-zero status. This prevents the script from continuing to execute commands after an error has occurred, which could lead to unexpected behaviour or further errors.
   Keep in mind that simple operations like variable++ might not have an exit code of 0 and stop execution of your script.
2. set -u: This option treats unset variables as an error and exits immediately. This helps catch typos and other mistakes where a variable is referenced before being set.
3. set -x: Print each command and its arguments to standard error as they are executed, which helps in debugging. Keep in mind that combined with the azure pipelines flag to exit on stderr, this might lead to false positives in error detection in your pipeline. It might also spam your logs with more information than you expected when looping in your script.
4. set -o pipefail: This option ensures that the return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status. This prevents errors in a pipeline from being masked.
5. IFS=$'\n\t': This sets the Internal Field Separator (IFS) to newline and tab only, which helps avoid issues with word splitting on spaces. The default IFS includes space, tab, and newline, which can lead to unexpected splitting of strings. Keep in mind that you don't need this flag if you never split strings. If you do, have a look at your data and figure out the best field separators for you.

Always remember that you can choose any of the switches to set in combination. You can leave out parts that might cause more trouble than they might help. Don't blindly copy the two lines into your pipelines and scripts, but understand the implications of them.
For general usage, the two lines above are still a good fit and prevent most errors.

Combining Unofficial Bash Strict Mode with Azure Pipelines

To integrate the unofficial bash strict mode into your Azure Pipelines scripts, you can include the necessary settings at the beginning of your bash scripts. Here’s an example of how to do this:

pool:
  vmImage: 'ubuntu-latest'

steps:
- script: |
    set -euxo pipefail
    IFS=$'\n\t'

    # Your script commands here
    echo "Running in strict mode"
  displayName: 'Run script with unofficial bash strict mode'

That's all Folks!

Integrating the unofficial bash strict mode into your Azure Pipelines scripts can lead to more robust, reliable, and maintainable automation. By ensuring that errors are caught early and providing detailed debugging information, you can streamline your CI/CD processes and reduce the likelihood of subtle bugs causing issues in your deployments.

As always, it is worth understanding what each of the toggles and knobs do and choosing what you want to implement and why. The unofficial bash strict mode isn't a wonder weapon, but a nice little toolset from which you can pick from.