{"id":2878,"date":"2019-09-12T14:44:35","date_gmt":"2019-09-12T02:44:35","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=2878"},"modified":"2019-09-12T16:41:08","modified_gmt":"2019-09-12T04:41:08","slug":"azure-devops-and-delphi-template-for-builds-running-tests-and-capturing-results","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/2878\/","title":{"rendered":"Azure DevOps – Template for Builds + Running Tests and Capturing Results"},"content":{"rendered":"[Estimated Reading Time: <\/span> 9<\/span> minutes]<\/span><\/span>\n

Last time we established that Azure DevOps is able to build Delphi code on my self-hosted build agent, but the script we used was pretty basic. Time to look at a more sophisticated script and use pipeline templates to re-use that script where-ever I need to build a Delphi project. We’ll also look at running tests and capturing results for presentation by Azure DevOps.<\/p>\n\n\n\n\n\n\n\n

Before jumping into the template itself, let’s first take a look at the way we use a template in a consuming pipeline:<\/p>\n\n\n\n

trigger:\n- develop\n- master\n\npool:\n  name: 'The Den'\n  demands: 'Delphi'\n\n- powershell: |\n    $GitVersion = GitVersion | ConvertFrom-Json\n    Write-Host "##vso[task.setvariable variable=version]$($GitVersion.SemVer)"\n    Write-Host "##vso[build.updatebuildnumber]$($GitVersion.SemVer)"\n  displayName: Determine version for build\n\n- script: |\n    SET USES=.\\fastmm-4.99.1\n    SET USES=%USES%;..\\..\\src\n    SET USES=%USES%;.\\deltics.rtl\n    SET USES=%USES%;.\\deltics.inc\n    ECHO ##vso[task.setvariable variable=uses]%USES%\n  displayName: 'Initial setup for building and running tests'\n\n- template: templates\/delphi-build.yml\n  parameters:\n    delphiVersion: 7\n    project: tests\\projects\\selftestD7\n    searchPath: $(uses)\n    postBuild:\n      - script: tests\\projects\\.bin\\selftestD7.exe -f=tests\\projects\\.results\\delphi.7.xml\n        displayName: 'Execute tests'\n\n- template: templates\/delphi-build.yml\n  parameters:\n    delphiVersion: xe4\n    project: tests\\projects\\selftest.XE4\n    searchPath: $(uses)\n    postBuild:\n      - script: tests\\projects\\.bin\\selftest.xe4.exe -f=tests\\projects\\.results\\delphi.xe4.xml\n        displayName: 'Execute tests'\n        \n- task: PublishTestResults@2\n  inputs:\n    testResultsFormat: 'XUnit'\n    testResultsFiles: '*.xml'\n    searchFolder: 'tests\/projects\/.results'\n    testRunTitle: '$(version) Test Results'<\/code><\/pre>\n\n\n\n
This is a simplified version of the actual pipeline I’m using.  In the full pipeline I build for every version of Delphi on my agent.  I’ve reduced this to just Delphi 7<\/strong> and Delphi XE4<\/strong> for the purposes of this example (to show old and new-ish).<\/p>\n\n\n\n
We’ve also now progressed beyond the Hello Delphi<\/strong> project.  This pipeline builds and runs the self-tests for my Smoketest<\/a><\/strong> unit testing framework<\/a> so some of the pre-amble in this script relates to that, as we’ll see.<\/p>\n\n\n\n
Let’s walk through it…<\/p>\n\n\n\n
The first powershell<\/strong> step runs GitVersion<\/a><\/strong> which will determine an appropriate version with which to label any build based on the git commit history to the point at which this build runs.  The results of this are output in json<\/strong> format which I capture into a variable, $GitVersion<\/strong>.<\/p>\n\n\n\n
I then execute two build commands<\/em>.<\/p>\n\n\n\n
These take the form of output echoed to the build agent which will be picked up by the build agent and interpreted as commands to be performed by the build agent on our behalf.  In a Powershell script we use Write-Host<\/strong> to output these commands.  In a cmd.exe<\/strong> script we would use ECHO<\/strong>.<\/p>\n\n\n\n
It is also possible to create entirely customised pipeline tasks using a provided Api<\/a>, with even greater access to controlling the build agent and interacting with the build pipeline.  But these console commands are a neat way to expose some of that functionality to any pipeline script, without requiring you to get your hands dirty with custom task development.<\/p>\n\n\n\n
The first build command sets a variable which can be picked up in later steps in this build.  This variable simply captures the SemVer<\/a> representation of the build version, as determined by GitVersion:<\/p>\n\n\n\n
Write-Host "##vso[task.setvariable variable=version]$($GitVersion.SemVer)"<\/code><\/pre>\n\n\n\n
Why not just keep using the $GitVersion<\/strong> variable itself ?<\/p>\n\n\n\n
Well, I can do that in this<\/em> script, but later steps will run in separate processes and so won’t be aware of this variable.  By setting a variable on the agent itself, via the build agent command, the variable can be referenced in later steps and is actually substituted into those steps by the build agent, so this works if those steps are not script-based and cannot directly make use of environment or other script variables.<\/p>\n\n\n\n
The next build command is passed back by the build agent to the Azure DevOps build process itself and labels the currently running build with the specified version:<\/p>\n\n\n\n
Write-Host "##vso[build.updatebuildnumber]$($GitVersion.SemVer)"\n  displayName: Determine version for build<\/code><\/pre>\n\n\n\n
The initial ##vso<\/strong> in the output is what marks these out as one of these special commands.  There are many<\/em> different commands available<\/a>.<\/p>\n\n\n\n
The next step is a simple cmd.exe<\/strong> script which uses an environment variable to build a search path that is needed by my selftest project.  Using the same ##vso<\/strong> command, I stash this in a build variable which I call uses<\/strong>, though this time ECHO<\/strong> is used since this is a cmd.exe<\/strong> script:<\/p>\n\n\n\n
ECHO ##vso[task.setvariable variable=uses]%USES%<\/code><\/pre>\n\n\n\n
The next two steps build the self-test project for each of Delphi 7<\/strong> and XE4<\/strong>.  These steps use a build template (which we’ll look at next).  First let’s look at how I use the template to run tests.<\/p>\n\n\n\n
Build templates support parameters to configure them for a particular use, and here I’m using a postBuild<\/strong> parameter to specify an additional script to run after the build (only if it is successful).  Here’s that postBuild<\/strong> script as specified for the XE4 build:<\/p>\n\n\n\n
- template: templates\/delphi-build.yml\n  parameters:\n    delphiVersion: xe4\n    project: tests\\projects\\selftest.XE4\n    searchPath: $(uses)\n    postBuild:\n      - script: tests\\projects\\.bin\\selftest.xe4.exe -f=tests\\projects\\.results\\delphi.xe4.xml\n        displayName: 'Execute tests'<\/code><\/pre>\n\n\n\n
I am relying here on the knowledge that my build template places any resulting .exe<\/strong> in a .bin<\/strong> folder which it will create under the project path.  It also creates a .results<\/strong> folder to hold any test results.<\/p>\n\n\n\n
Since the self-test project is itself a Smoketest<\/a><\/strong> suite, the results of the tests can be output in a file specified on the command line that runs the tests.  In this case I put the results from each run into that .results<\/strong> folder and since for this project I am running tests for each version of Delphi that I build with, I give those results a name that identifies the version of Delphi to which the test results relate.<\/p>\n\n\n\n
The final step runs a standard task from the Azure DevOps task library to scan for test results in a specified location (this can be very generic and find any test results produced during a run, but in this case I know where the results are so can direct the scan to that specific location).<\/p>\n\n\n\n
- task: PublishTestResults@2\n  inputs:\n    testResultsFormat: 'XUnit'\n    testResultsFiles: '*.xml'\n    searchFolder: 'tests\/projects\/.results'\n    testRunTitle: '$(version) Test Results'<\/code><\/pre>\n\n\n\n
Any results found are sent back by this task to the Azure DevOps host where they will be ingested into the results for this build.  So now, as well as metrics for the build:<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
I can now also view and dig into test results:<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
This works because I have recently re-jiggered the output from Smoketest<\/strong> to output results in xUnit 2<\/strong> compatible format (though this is not yet in the publicly available version of Smoketest<\/strong>).<\/p>\n\n\n\n
\n\n\n\n
Now let’s take a look at the build template itself:<\/p>\n\n\n\n
parameters:\n  delphiVersion: ''\n  project: ''\n  platform: ''\n  searchPath: ''\n  unitScopes: 'System;System.Win;Vcl;WinApi'\n  fixPack: 'true'\n  verbose: 'false'\n  runIfSuccessful: 'true'\n  postBuild: []\n\nsteps:\n- powershell: |\n    $delphi='${{ parameters.delphiVersion }}'\n    $project='${{ parameters.project }}'\n    $searchPath='${{ parameters.searchPath }}'\n    $unitScopes='${{ parameters.unitScopes }}'\n    $noFixPack='${{ parameters.noFixPack }}'\n    $verbose='${{ parameters.verbose }}'\n    $runIfSuccessful='${{ parameters.runIfSuccesful }}'\n\n    $bits = if($platform -eq 'x64') {'64'} else {'32'}\n\n    $dpr=$(Split-Path -Leaf $project) + '.dpr'\n    $cfg='dcc' + $bits + '.cfg'\n    $exe='.bin\\' + $(Split-Path -Leaf $project) + '.exe'\n    $delphiRoot='c:\\dcc\\' + $delphi\n    $delphiBin=$delphiRoot + '\\bin'\n    $delphiLib=$delphiRoot + '\\lib'\n        \n    # Assume that we will use the standard Delphi compiler but then\n    #  check for the presence of an appropriate IDE FixPack compiler\n    #  (unless the fixPack parameter is anything other than 'true').\n    #\n    # If an IDE FixPack compiler is found, use that instead.\n\n    $dcc = 'dcc' + $bits\n    if($fixPack -eq 'true') {\n        Write-Host 'Checking for IDE FixPack compilers...'\n\n        if(Test-Path -Path $($delphiBin + '\\dcc' + $bits + 'speed.exe')) {\n            $dcc = 'dcc32'\n        } elseif(Test-Path -Path $($delphiBin + '\\fastdcc' + $bits + '.exe')) {\n            $dcc = 'fastdcc' + $bits\n        }\n    }\n    $dcc=$delphiBin + '\\' + $dcc + '.exe'\n    Write-Host $('Compiling with ' + $dcc)\n\n    # Modify the Delphi Lib path according to the specified Delphi version\n    #  and insert into the searchPath\n\n    if($delphi -eq 'xe') {\n        $delphiLib=$delphiLib + '\\win32\\release'\n    } elseif($delphi -gt 'xe') {\n        $delphiLib=$delphiLib + '\\win' + $bits + '\\release'\n    }\n    $searchPath=$delphiLib + ';' + $searchPath\n    Write-Host $('Delphi library path ' + $delphiLib)\n    Write-Host $('Using search path ' + $searchPath)\n        \n    # Construct compiler options for the dccNN.cfg file then create that file\n\n    Set-Location (Split-Path -Path $project)\n\n    if(-Not(Test-Path -Path .bin)) { New-Item .bin -ItemType directory | Out-Null }\n    if(-Not(Test-Path -Path .results)) { New-Item .results -ItemType directory | Out-Null }\n\n    $optD='-DCONSOLE'\n    $optE='-E.bin'\n    $optI='-I' + $searchPath\n    $optN='-N.bin'\n    $optR='-R' + $searchPath\n    $optU='-U' + $searchPath\n\n    if($delphi -gt 'xe') {\n        $optNS='-NS' + $unitScopes\n    }\n        \n    Write-Host $('Creating compiler configuration ' + $cfg)\n\n    if(Test-Path -Path $cfg) { Remove-Item $cfg | Out-Null }\n    New-Item $cfg -ItemType file | Out-Null\n\n    Add-Content $cfg $optD\n    Add-Content $cfg $optE\n    Add-Content $cfg $optI\n    Add-Content $cfg $optN\n    Add-Content $cfg $optR\n    Add-Content $cfg $optU\n    if($delphi -gt 'xe') {\n        Add-Content $cfg $optNS\n    }\n\n    # Remove any previous build and invoke the command-line compiler\n        \n    if(Test-Path -Path $exe) { Remove-Item $exe | Out-Null }\n\n    $cmd=$dcc + ' ' + $dpr + ' -CC'\n    Write-Host $('Compiling with ' + $cmd)\n    if($verbose -eq 'true') {\n        Invoke-Expression $cmd\n    } else {\n        Invoke-Expression $cmd | Select-String -Pattern 'Fatal: ','Hint: ','Warning: '\n    }\n\n    if(Test-Path -Path $exe) {\n        Write-Host $('Build succeeded! :)  [' + $exe + ']')\n    } else {\n        Write-Host "##vso[task.logissue type=error]Build failed.  :("\n        exit 1\n    }\n  displayName: 'Delphi ${{ parameters.delphiVersion }} Build of ${{ parameters.project }} '\n- ${{ parameters.postBuild }}<\/code><\/pre>\n\n\n\n
Most of this is the powershell <\/strong>build script that does the actual build, which is much more sophisticated than the simple test we saw last time<\/a>.<\/p>\n\n\n\n
Starting at the beginning, the initial parameters<\/strong> section identifies the parameters which may be passed in to this template, as we saw in the build itself.  The most interesting of these is the postBuild<\/strong> parameter which as we saw accepted an entire script to be run by this template.  But we can see here that this is not difficult to achieve – far from it in fact.<\/p>\n\n\n\n
The first thing that the build script does is place most of these parameters into script variables.  This isn’t strictly necessary, but has some benefits.<\/p>\n\n\n\n
  1. It does allow us to manipulate the values if we wish.  Parameter values are directly substituted into the script before it runs – i.e. you cannot treat parameters themselves as variables.<\/li>
  2. (and mainly) I’m doing this just to make it slightly less cumbersome to reference them later in the script. \ud83d\ude42<\/li><\/ol>\n\n\n\n
    What then follows is some further setup of script variables, driven by the version of Delphi identified for the build.  The aim here is to setup variables that identify the compiler executable (in the relevant bin<\/strong> folder) and the path to the VCL and RTL units that will be needed for the build (in the corresponding lib<\/strong> folder).<\/p>\n\n\n\n
    As I mentioned, my build agent has the bin<\/strong> and lib<\/strong> folders for many different versions of Delphi on it.  By following some conventions I’m able to construct paths to those folders by dropping in the Delphi version as required, also taking account of the target platform in later Delphi versions (to distinguish between x86<\/strong> and x64<\/strong> builds).<\/p>\n\n\n\n
    It should be fairly obvious how this is achieved by looking at a fragment of the file system as it is found on the build agent:<\/p>\n\n\n\n
    c:\\dcc\\7\n         \\bin\n            \\dcc32.exe\n         \\lib\n c:\\dcc\\xe4\n          \\bin\n             \\dcc32.exe\n             \\dcc64.exe\n          \\lib\n             \\win32\n                  \\debug\n                  \\release\n             \\win64\n                  \\debug\n                  \\release<\/pre>\n\n\n\n
    The next step is a little bit of finesse.<\/p>\n\n\n\n
    Alongside the stock compilers, in each bin folder I have also put copies of the “speed” and “fast” compilers from the amazing Andreas Hausladen<\/a>.  The build template checks for these, and if found will invoke those in preference to the stock compiler, to eek a little extra efficiency for my builds.<\/p>\n\n\n\n
    There is then a final tweak to the libPath<\/strong> for later Delphi versions before the .bin<\/strong> and .results<\/strong> folders are created in the project location.<\/p>\n\n\n\n
    The script then constructs a compiler configuration file.  I anticipate this is a section that will either be replaced in the future with something driven by project configuration or will grow to a point where reliance on those project files is avoided entirely.<\/p>\n\n\n\n
    There are arguments both ways as to which is the more desirable.<\/p>\n\n\n\n