Automated testing plays a central role in ensuring the quality and reliability of software deliveries. However, automating API tests within CI/CD workflows remains a challenge for many teams. The CLI client Bruno offers an innovative solution to simplify this task. This open-source tool allows for fast, efficient, and integrated execution of API tests. In this article, we’ll explore how to configure a CI/CD pipeline in Azure DevOps to automate these tests.
Test automation helps prevent regressions and ensures that APIs function correctly. Here are the main challenges encountered when integrating these tests into a CI/CD workflow:
Bruno, an alternative to Postman, is an open-source tool designed specifically for testing REST APIs. It stands out for its performance and simplicity, providing a smooth and efficient experience. Its graphical interface includes all essential features of an HTTP client, ensuring intuitive usage. Additionally, its Tests tab makes it easy to define assertions, streamlining the request validation process.
What really sets it apart is the ability to store API requests as code within project files. This approach offers full control over API requests and tests, facilitating sharing, change tracking, and collaboration within technical teams—especially thanks to its seamless integration with Git.
For more information, see the official documentation.
Bruno is known for its ease of use, offering intuitive syntax and commands that integrate well via its CLI. It also brings flexibility by integrating seamlessly into PowerShell scripts and CI/CD pipelines, making it a great fit for development workflows.
That said, using Bruno may require some initial learning, especially due to the specific structure of its test collections, which may take time for new users to get familiar with.
In Azure DevOps, we created a Bruno Test Task Group (BT) that enables API operations using the Bruno CLI. This Task Group’s parameters include essential information such as the API Key, the authentication scope, the Bruno project root path, and the test request folders.
The Task Group is structured into four steps:
The first step is to retrieve secrets stored in Azure Key Vault, such as clientID and clientSecret, which are required to authenticate API calls. We use Azure CLI for this, allowing secrets to be accessed directly from the command line:
$Secret = az keyvault secret show --name "secret-key-name" --vault-name "key-vault-service-name" --query "value" -o tsv
Once the secret is retrieved, it’s stored in an Azure DevOps pipeline variable to be reused in the next steps:
Write-Host "##vso[task.setvariable variable=Secret]$Secret"
To ensure secure access, it’s best to grant permissions to the Service Principal associated with Azure DevOps on the Key Vault rather than exposing a Personal Access Token (PAT) in the pipeline. This maintains high security while avoiding permission issues.
Before running tests, Node.js and Bruno CLI must be installed on the runner agent using:
npm install -g @usebruno/cli
Tests are executed from the Bruno project root ($path) by specifying the folder containing the operations to be tested ($folder). For correct Azure DevOps pipeline execution, manage the environment configuration file ($configFilePath) and make sure test reports are correctly generated in ($destinationReport).
Name the environment file based on the current pipeline environment ($(Env_Maj)), stored in an Azure DevOps variable library.
This approach supports multiple deployment environments.
$folder = "$(QueriesFolder)" $path = "$(ProjectPath)" $configFilePath = "$path\environments\$(Env_Maj).bru" $destinationReport = "$path\Results\report.xml"
Before executing API tests, it is essential to inject the required variables dynamically into the <ENV>.bru configuration file. The PowerShell script below handles this by reading the existing file, modifying or adding the vars section (API host, credentials, client ID, etc.), and saving the updated file.
This ensures test consistency by aligning variable values with those defined in the pipeline.
If the config file doesn’t exist, the script generates a base one to prevent errors.
Error handling is included to guarantee process reliability by validating file access and permissions. This method allows fully automated and environment-specific configuration updates, eliminating manual intervention.
# Read, modify, and save the configuration file
Write-Host "Updating configuration file: $configFilePath"
if (Test-Path $configFilePath) {
$existingContent = Get-Content -Path $configFilePath -Raw
} else {
$existingContent = ''
Write-Warning "Configuration file '$configFilePath' not found. A new file will be created."
}
$newVarsSection = @"
vars {
apimHost: $(Common_Host_Name)
scope: $(Scope)
apiKeyExternal: $subscriptionValue
client_id: $clientIdValue
client_secret: $clientSecretValue
}
"@
$updatedContent = $existingContent -replace '(?s)vars:secret\s*\[.*?\]', ''
if ($existingContent -match '(?s)vars\s*{.*?}') {
$updatedContent = $updatedContent -replace '(?s)vars\s*{.*?}', $newVarsSection
Write-Host "Existing 'vars {}' section replaced."
} else {
$updatedContent = $updatedContent + "`r`n" + $newVarsSection
Write-Host "New 'vars {}' section added."
}
try {
Set-Content -Path $configFilePath -Value $updatedContent -Force
Write-Host "Configuration file successfully saved: $configFilePath"
} catch {
Write-Error "Failed to update configuration file. Check permissions and path."
exit 1
}
Once the configuration file is updated, run the tests using the following command. The --output and --format options are used to generate a JUnit-style test report:
bru run --env "$(Env_Maj)" --output "$destinationReport" --format junit $folder
The XML files containing test results are published using Publish Test Results from the path $destinationReport.
By integrating this step into the pipeline, the team ensures that test reports are correctly processed and available.
Once published, results appear under the Tests tab in Azure DevOps, showing execution history and details for any failed tests.
Integrating Bruno into an Azure DevOps pipeline brings real benefits to API testing.
By automating tests, teams save time on manual validations while ensuring greater deployment reliability.
Embedding requests directly into code simplifies versioning and fits naturally into developer workflows.
To ensure a secure and efficient pipeline, several best practices are critical:
In practice, this approach saves time by reducing repetitive tasks and minimizing human error.
Automated tests ensure greater reliability across all deployment environments.
Some improvements are still possible, such as improving test data set management to ensure stable and reproducible tests.
Likewise, customizing reports with specific metadata could further enhance result analysis.
Adopting Bruno in a CI/CD pipeline is a step toward robust, high-performance automation that improves API test quality.