0

I am attempting to use Azure Pipelines to deploy multiple copies of the same code base with variations in the configuration for each instance. Each one is to be deployed to a separate domain registered in IIS on my server. Currently I have the separate configurations set up as a part of the pipeline hardcoded as parameters.

# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4

trigger:
  branches:
    include:
      - <triggerBranch>

pool:
  name: '<poolName>'
  vmImage: 'windows-latest'

parameters:
  #Define the Sales Partner Instances to use
  - name: salesPartners
    type: object
    default:      
      - name: "SalesPartner1"
        salesPartnerName: "SalesPartner1"
        domain: "<domain>"
        id: "<Id>"
        thumbprint: "<thumbprint>"
        applicationTheme: "<themePath>"
      - name: "SalesPartner2"
        salesPartnerName: "SalesPartner2"
        domain: "<domain>"
        id: "<Id>"
        thumbprint: "<thumbprint>"
        applicationTheme: "<themePath>"
      - name: "SalesPartner3"
        salesPartnerName: "SalesPartner3"
        domain: "<domain>"
        id: "<Id>"
        thumbprint: "<thumbprint>"
        applicationTheme: "<themePath>"
variables:
  azSub: '<serviceConn>'
  testKeyVault: '<value>'
  uatKeyVault: '<value>'
  prodKeyVault: '<value>'
  buildPlatform: 'AnyCPU'
  buildConfiguration: 'Release'
  solution: '<value>'
  project: '<value>'
  projectName: '<value>'
  webAppName: '<value>'
  webAppPhysicalPath: '%SystemDrive%\inetpub\$(webAppName)'
  resellerEndpoint: '<value>'
stages:
  - stage: Build
    jobs:
      - job:
        displayName: "Build"
        steps:

          - script: |
              echo Build.BuildNumber is $(Build.BuildNumber)
              echo Build.BuildID is $(Build.BuildID)
              echo Build.SourceBranch is $(Build.SourceBranch)
              echo Build.SourceBranchName is $(Build.SourceBranchName)
              echo Build.SourceVersion is $(Build.SourceVersion)

          - task: NuGetToolInstaller@1
            name: 'NuGetToolInstaller'
            displayName: 'Nuget Tool Installer'
          
          - task: NuGetCommand@2
            name: 'NuGetRestore'
            displayName: 'Restore NuGet packages.'
            inputs:
              restoreSolution: '$(solution)'
                  
          - task: VSBuild@1
            name: 'BuildProject'
            displayName: 'Build the project.'
            inputs:
              solution: '$(project)'
              msbuildArgs: '/p:DeployOnBuild=true
                            /p:WebPublishMethod=Package
                            /p:PackageAsSingleFile=true
                            /p:SkipInvalidConfigurations=true
                            /p:TransformWebConfigEnabled=false
                            /p:AutoParameterizationWebConfigConnectionStrings=false
                            /p:PackageLocation="$(build.artifactStagingDirectory)"'
              platform: '$(buildPlatform)'
              configuration: '$(buildConfiguration)'
          
          - task: PublishBuildArtifacts@1
            name: 'PublishBuildArtifact'
            displayName: 'Publish build artifact.'
            inputs:
              PathtoPublish: '$(Build.ArtifactStagingDirectory)'
              ArtifactName: '$(webAppName)'
              publishLocation: 'Container'
  - stage: RetrieveResellers
    displayName: 'Retrieve Resellers'
    dependsOn: Build
    condition: succeeded()
    jobs:
      - job: FetchResellers
        displayName: 'Fetch Resellers from API'
        steps:
        - task: PowerShell@2
          displayName: 'Get Resellers'
          inputs:
            targetType: 'inline'
            script: |
              # Make your API call
              $response = Invoke-RestMethod -Uri $(resellerApiEndpoint) -Method Get
          
              # Save the response as JSON file
              $response | ConvertTo-Json -Depth 10 | Out-File -FilePath "$(Build.ArtifactStagingDirectory)/salesPartners.json" -Encoding UTF8
          
              # Optional: Display what we got for debugging
              Write-Host "Retrieved $($response.Count) sales partners:"
              foreach ($partner in $response) {
                Write-Host "  - $($partner.salesPartnerName) ($($partner.domain))"
              }
        - task: PublishBuildArtifacts@1
          displayName: 'Publish deployment configuration'
          inputs:
            PathtoPublish: '$(Build.ArtifactStagingDirectory)/salesPartners.json'
            ArtifactName: 'deployment-config'

  - stage: DeployToTest
    displayName: 'Deploy to Test'
    dependsOn: RetrieveResellers
    condition: succeeded()
    jobs:
      #Run the Deployment Stage for each instance
      - ${{ each salesPartner in parameters.salesPartners }}:
        - deployment:
          environment:
            name: 'Test'
            resourceType: VirtualMachine
            tags: IIS, SalesViewWebApp
          variables:
            Release.EnvironmentName: '$(Environment.Name)'
            GravityDb: '$(ConnectionStrings-GravityDb)'
            GravityDocuments: '$(ConnectionStrings-GravityDocuments)'
            PayrollDb: '$(ConnectionStrings-PayrollDb)'
            TransactionDb: '$(ConnectionStrings-TransactionDb)'
            PCIControlScanDb: '$(ConnectionStrings-PCIControlScanDb)'
            ArtefactsDb: '$(ConnectionStrings-ArtefactsDb)'
            GlobalPaymentsDb: '$(ConnectionStrings-GlobalPaymentsDb)'
            FirstDataDb: '$(ConnectionStrings-FirstDataDb)'
            ReportingDb: '$(ConnectionStrings-ReportingDb)'
            elmah: '$(ConnectionStrings-elmah)'
            environment: '$(Environment.Name)'
            AccessOneAPIUsername: '$(AppSettings-AccessOneAPIUsername)'
            AccessOneAPIToken: '$(AppSettings-AccessOneAPIToken)'
            ResellerId: '${{ salesPartner.id }}'
            Domain: '${{ salesPartner.domain }}'
            BrandName: '${{ salesPartner.name }}'
            ApplicationThemeHref: '${{ salesPartner.applicationTheme }}'
          strategy:
            runOnce:
              deploy:
                steps:
                  - task: AzureKeyVault@1
                    inputs:
                      azureSubscription: '$(azSub)'
                      KeyVaultName: '$(testKeyVault)'
                      SecretsFilter: '*'
                      RunAsPreJob: true      
                  - task: IISWebAppManagementOnMachineGroup@0
                    inputs:
                      IISDeploymentType: IISWebsite
                      ActionIISWebsite: CreateOrUpdateWebsite
                      WebSiteName: '${{ salesPartner.salesPartnerName}}'
                      WebsitePhysicalPath: '%SystemDrive%\inetpub\${{ salesPartner.salesPartnerName }}'
                      WebsitePhysicalPathAuth: 'WebsiteUserPassThrough'
                      AddBinding: false
                      CreateOrUpdateAppPoolForWebsite: true
                      AppPoolNameForWebsite: '${{ salesPartner.salesPartnerName }}'
                      AppPoolName: '${{ salesPartner.salesPartnerName }}'
                      DotNetVersionForWebsite: 'v4.0'
                      PipeLineModeForWebsite: 'Integrated'
                      AppPoolIdentityForWebsite: ApplicationPoolIdentity 
                  - task: PowerShell@2
                    displayName: Add Binding
                    continueOnError: true
                    inputs:
                      targetType: 'inline'
                      script: |
                        Import-Module WebAdministration
                        function Check-IISBinding {
                        param(
                          [string]$SiteName,
                          [string]$Protocol,
                          [string]$HostName,
                          [int]$Port,
                          [string]$IpAddress
                        )
                        Write-Host "Site Name - $SiteName | Protocol - $Protocol | IPAddress = $IpAddress | Port - $Port | HostHeader - $HostName"
                                                  
                        $binding = [bool](Get-WebBinding -Name $SiteName -IPAddress $IPAddress -Port "$Port" -HostHeader $HostName -Protocol $Protocol)
                        Write-Host "Binding $binding"
                        if ($binding) {
                            Write-Host "Binding '$Protocol $BindingInformation' already exists for site '$SiteName'."
                            return $true
                        } else {
                            Write-Host "Binding '$Protocol $BindingInformation' does not exist for site '$SiteName'."
                            return $false
                        }
                        }
                        $siteName = "${{ salesPartner.salesPartnerName }}"
                        $protocol = "https"
                        $hostName = "${{ salesPartner.domain }}"
                        $ipAddress = "*"
                        $thumbprint = "${{ salesPartner.thumbprint }}"
                        $port = 443
                        # Check if the binding exists before attempting to add it
                        if (!(Check-IISBinding -SiteName $siteName -Protocol $protocol -HostName $hostName -Port $port -IpAddress $ipAddress)) {
                          # Add the binding if it doesn't exist
                          New-WebBinding -Name $siteName -IPAddress $ipAddress -Port $port -HostHeader $hostName -Protocol $protocol
                          (Get-WebBinding -Name $siteName -IPAddress $ipAddress -Port "$port" -HostHeader $hostName -Protocol $protocol).AddSslCertificate($thumbprint, "WebHosting")
                          Write-Host "Binding '$protocol 0.0.0.0:443:$hostName ' added to site '$siteName'."
                        } else {
                          Write-Host "Binding '$protocol 0.0.0.0:443:$hostName' already exists for site '$siteName'."
                        }
                  - task: IISWebAppDeploymentOnMachineGroup@0
                    inputs:
                      WebSiteName: '${{ salesPartner.salesPartnerName }}'
                      Package: '$(Pipeline.Workspace)\$(webAppName)\$(projectName).zip'
                      RemoveAdditionalFilesFlag: true
                      ExcludeFilesFromAppDataFlag: true
                      TakeAppOfflineFlag: true
                      XmlTransformation: true
                      XmlVariableSubstitution: true

The current solution works well; it deploys the separate sites and performs the desired configuration transformation based on the parameters provided. However, I want the pipeline to be more scalable, and having to add a new set of values manually everytime a new partner is onboarded isn't realistic long term. I created an API endpoint as an Azure function to retrieve the list of Sales Partner configuration information from the database, which is referenced in the stage "RetrieveResellers". Currently, the results of that api call are being saved as a json file in the Build artefact directory.

I would like to be able to use the results of the API call as pipeline configuration, but I've run into a couple of issues with it: There doesnt appear to be a good way in the already used Pipeline tasks to access the JSON file once it has been saved.

Is there a strategy I can use to access this information dynamically rather than having to hard-code configure the pipeline for every instance?

0

1 Answer 1

0

The ${{ <exp> }} expressions are evaluated at compile-time. There aren't any extensions to the Azure DevOps pipeline runtime that would allow you to dynamically fetch data from an external service and inject that data into the current pipeline during it's compilation stage.

However, looking at your pipeline, you are defining the salesPartners as a parameter which influences the pipeline's compile-time evaluations. The Pipeline API has a Run Build endpoint that has the ability to specify the template parameters as part of the POST body, or even provide your own YAML. We can use this.

If you want the deployment stages to be dynamically defined at runtime, I would break your existing pipeline into two separate pipelines. One for continuous-integration (build), the second for continuous-delivery (deployment).

At the tail-end of your build pipeline, query your function app for the list of resellers and then use the Run Build endpoint to pass the salesPartners template parameter to the second pipeline.

For example, assuming the reseller information coming back from your function app has the exact structure of your yaml object in JSON format:

- job: queue_deployment_pipeline
  displayName: Fetch Resellers and Queue Deployment
  condition: ne(variables['Build.BuildReason'], 'PullRequest')
  steps:
  - pwsh: |
      $resellers = Invoke-RestMethod ... | ConvertFrom-Json

      $pipelineId = 12345; # id of your cd pipeline

      $baseUrl = "$(System.CollectionUri)/$(System.TeamProjectId)"
      $api = "/_apis/pipelines/$pipelineId/runs?api=version=7.1"
      $token = $env:SYSTEM_ACCESSTOKEN
      $headers = @{ Authorization = "Bearer $token" }

      $body = @{
         templateParameters = @{
            salesPartners = $resellers
         }
      } | ConvertTo-Json -Depth 10

      Invoke-RestMethod `
         -Uri $baseUrl + $api `
         -Method Post `
         -Body $body

This obviously assumes that the agent being used has been granted permissions to queue the pipeline.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.