Skip to content

Commit f23050d

Browse files
andr317candr317c
authored
V16 QA cherry picked acceptance tests with different configuration (#20106)
* Added appsettings * Added test setup for different config * Added appsettings for external login * Added acceptance tests * Updated pipelines * Updated solution file * V15 QA Added external login provider tests and split pipeline into templates (#20049) * Added setup for external login * Started on yaml * Added test file * Updated pipeline * Use env vars * Added env variables and commented out test we don't need to run * Removed list from matrix * Updated condition * Updated package path * Updated testFolder * double slash * Updated condition * Updated condition again * Added port * Removed redundant values * Set as env vars * Added env vars beneath matrix * Get env * Updated naming * Updated usage of values * Added a check for client id, to see if value set * Moved env out of pool * Tried moving env * Trying to fix the env being empty * Removed env * Updated name of variable * Fixed A cyclical reference * Updated typo * More logging * Reverted change * Added env * Added env to tests * Cleaned up * Added yaml template files * Updated nighly pipepline to use templates * Updated sln * Split yaml into templates for e2e setup * Updated pipeline * Updated solution file * Set value * Added if statement * Added variables * Set default values * Updated values * Updated condition * Run multiple tests * Added env * Updated from parameter to variable * Fixed condition * Fixed condition to use actual value * Updated npx wait on command * Updated pwsh * Updated port again * Updated port value * Updated wait on * Updated condition * Restructured * Updated var * Updated run application steps * Added echo * Updated to boolean * Updated conditions * Updated test template usage * Added databaseType * Added another databaseType * Split up templates * Fixed indentation * Updated condition * updated path * removed build from path * Updated conditions for azureAd * Fixed indentation * Updated to single qoutes * Cleaned up * Removed unused file * Clarified namin * Moved * Updated pipeline, not done * Updated locator * Updated pipelines * Updated test helpers package * Skipped build stage for default app settings tests * Updated password var * Updated locators * Updated defaultconfig build setup * Split E2E stage in two * Added parameter for skipping integration tests * Cleaned up * Added ASPNETCORE_URLS * V15 QA acceptance tests with appsettings (#19550) * Start of appsetting * Updated setup of playwright * Adjusted the pipeline * Updated appsetting * Added install test * Added comments * Updated pipeline * Updated development app settings * Commented tests out * comment * Added if statement * Updated pipeline * Fixed condition * Changed to production * Added a log * Updated copy item * Added * Updated app settings * Updated pipeline * Moved playwright login * Updated pipeline * Updated app setting * Updated nightly * Updated appsettings * Updated get * Updated wait on * Updated appsettings * Updated connection string * Updates * Skips code * Updated variable * Updated pipeline * We want to always retain the trace, to see if the test runs as expected on the pipeline * Added a temporary wait till port is open * Fixed condition * Added missing tcp for wait on * Updated URL env * Updated setup * Fixed string * Updated locator * Split tests into SQLite and SQLServer * Updated pipeline to run all tests * Retain trace on failure * Added testFolder var * Added appsettings and program for delivery api tests * Updated playwright config * Split test runners into defaultconfig and different app settings * Added delivery api tests * Cleaned up tests * Bumped version * Updated pipeline * Small fixes * Added password * Updated connection string * Fixed * Removed quotes * Removed unnecessary connection string * Added missing password * Cleaned up * Cleaned up * Cleaned up * Updated to use helpers * Bumped version * Updated helper usage * Added password to variables and a condition * Added check * Indented value * Fixed condition * More updates * Updated variable * Removed settings * Updated delivery api tests * Bumped version * Updated test * Removed unnecessary variables * Updates based on copilot comments * Fixed merge conflict * Fixed env creation step * Bumped version * Updated tests to use new helper * Updated helper * Updated to string * Moved logic to conditions * bumped version * Use new name for helper * Remove echo * Added variable
1 parent fd0ccc5 commit f23050d

28 files changed

Lines changed: 1362 additions & 730 deletions

‎build/azure-pipelines.yml‎

Lines changed: 71 additions & 435 deletions
Large diffs are not rendered by default.

‎build/nightly-E2E-build-template.yml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
parameters:
2+
- name: testFolder
3+
type: string
4+
default: ''
5+
6+
- name: buildConfiguration
7+
type: string
8+
default: ''
9+
10+
- name: additionalEnvironmentVariables
11+
type: string
12+
default: 'false'
13+
14+
steps:
15+
- pwsh: |
16+
dotnet restore UmbracoProject
17+
cp $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest.UmbracoProject/*.cs UmbracoProject
18+
displayName: Restore project
19+
workingDirectory: $(Agent.BuildDirectory)/app
20+
21+
# Update application to use necessary app settings
22+
- pwsh: |
23+
$sourcePath = "$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/tests/${{ parameters.testFolder }}/AdditionalSetup"
24+
$destinationPath = "UmbracoProject"
25+
$jsonFiles = Get-ChildItem -Path $sourcePath -Filter "*.json"
26+
if ($jsonFiles) {
27+
$jsonFiles | ForEach-Object {
28+
Write-Host "Copying: $($_.FullName)"
29+
Copy-Item -Path $_.FullName -Destination $destinationPath -Force
30+
}
31+
} else {
32+
Write-Host "No JSON files found."
33+
}
34+
displayName: Update application to use necessary app settings
35+
workingDirectory: $(Agent.BuildDirectory)/app
36+
37+
# Update application to use necessary App_Plugins
38+
- pwsh: |
39+
$sourcePath = "$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/tests/${{ parameters.testFolder }}/AdditionalSetup"
40+
$destinationPath = "UmbracoProject"
41+
$appPluginsFolders = Get-ChildItem -Path $sourcePath -Directory -Filter "App_Plugins"
42+
if ($appPluginsFolders) {
43+
foreach ($folder in $appPluginsFolders) {
44+
Write-Host "Copying folder: $($folder.FullName)"
45+
Copy-Item -Path $folder.FullName -Destination $destinationPath -Recurse -Force
46+
}
47+
} else {
48+
Write-Host "No App_Plugins found."
49+
}
50+
displayName: Update application to use necessary app plugins
51+
workingDirectory: $(Agent.BuildDirectory)/app
52+
53+
# Update application to use necessary classes
54+
- pwsh: |
55+
$sourcePath = "$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/tests/${{ parameters.testFolder }}/AdditionalSetup"
56+
$destinationPath = "UmbracoProject"
57+
$csharpFiles = Get-ChildItem -Path $sourcePath -Filter "*.cs"
58+
if ($csharpFiles) {
59+
$csharpFiles | ForEach-Object {
60+
Write-Host "Copying: $($_.FullName)"
61+
Copy-Item -Path $_.FullName -Destination $destinationPath -Force
62+
}
63+
} else {
64+
Write-Host "No C# files found."
65+
}
66+
displayName: Update application to use necessary classes
67+
workingDirectory: $(Agent.BuildDirectory)/app
68+
69+
- pwsh: |
70+
dotnet build UmbracoProject --configuration ${{ parameters.buildConfiguration }} --no-restore
71+
dotnet dev-certs https
72+
displayName: Build application
73+
workingDirectory: $(Agent.BuildDirectory)/app
74+
condition: and(succeeded(), eq(variables['additionalEnvironmentVariables'], 'false'))

‎build/nightly-E2E-run-application-template.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
parameters:
2+
- name: SA_PASSWORD
3+
type: string
4+
default: ''
5+
6+
- name: buildConfiguration
7+
type: string
8+
default: ''
9+
10+
- name: additionalEnvironmentVariables
11+
type: string
12+
default: 'false'
13+
14+
- name: DatabaseType
15+
type: string
16+
default: ''
17+
18+
steps:
19+
# Skips the SQLServer setup if the databaseType does not match
20+
- ${{ if eq(parameters.DatabaseType, 'SQLServer') }}:
21+
# Start SQL Server Linux
22+
- powershell: docker run --name mssql -d -p 1433:1433 -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=${{ parameters.SA_PASSWORD }}" mcr.microsoft.com/mssql/server:2022-latest
23+
displayName: Start SQL Server Docker image (Linux)
24+
condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
25+
26+
# Start SQL Server LocalDB Windows
27+
- pwsh: SqlLocalDB start MSSQLLocalDB
28+
displayName: Start SQL Server LocalDB (Windows)
29+
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
30+
31+
# Run application for Linux
32+
- bash: |
33+
nohup dotnet run --project UmbracoProject --configuration ${{ parameters.buildConfiguration }} --no-build --no-launch-profile > $(Build.ArtifactStagingDirectory)/playwright.log 2>&1 &
34+
echo "##vso[task.setvariable variable=AcceptanceTestProcessId]$!"
35+
displayName: Run application (Linux)
36+
condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux'), eq(variables['additionalEnvironmentVariables'], 'false'))
37+
workingDirectory: $(Agent.BuildDirectory)/app
38+
39+
# Run application for Windows
40+
- pwsh: |
41+
$process = Start-Process dotnet "run --project UmbracoProject --configuration ${{ parameters.buildConfiguration }} --no-build --no-launch-profile 2>&1" -PassThru -NoNewWindow -RedirectStandardOutput $(Build.ArtifactStagingDirectory)/playwright.log
42+
Write-Host "##vso[task.setvariable variable=AcceptanceTestProcessId]$($process.Id)"
43+
displayName: Run application (Windows)
44+
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'), eq(variables['additionalEnvironmentVariables'], 'false'))
45+
workingDirectory: $(Agent.BuildDirectory)/app

‎build/nightly-E2E-run-tests-template.yml‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
parameters:
2+
- name: ASPNETCORE_URLS
3+
type: string
4+
default: ''
5+
6+
- name: testCommand
7+
type: string
8+
default: ''
9+
10+
- name: port
11+
type: string
12+
default: ''
13+
14+
- name: AZUREB2CTESTUSEREMAIL
15+
type: string
16+
default: ''
17+
18+
- name: AZUREB2CTESTUSERPASSWORD
19+
type: string
20+
default: ''
21+
22+
- name: DatabaseType
23+
type: string
24+
default: ''
25+
26+
steps:
27+
# Ensures we have the package wait-on installed
28+
- pwsh: npm install wait-on
29+
displayName: Install wait-on package
30+
31+
# Wait for either the port of the aspnetcore url
32+
- pwsh: |
33+
$Port = "${{ parameters.port }}"
34+
$Url = "${{ parameters.ASPNETCORE_URLS }}"
35+
36+
if ($Port -ne "") {
37+
Write-Host "Waiting on TCP port $Port"
38+
npx wait-on -v --interval 1000 --timeout 120000 "tcp:$Port"
39+
} else {
40+
Write-Host "Waiting on URL $Url"
41+
npx wait-on -v --interval 1000 --timeout 120000 "$Url"
42+
}
43+
displayName: Wait for application
44+
workingDirectory: tests/Umbraco.Tests.AcceptanceTest
45+
46+
# Install Playwright and dependencies
47+
- pwsh: npx playwright install chromium
48+
displayName: Install Playwright only with Chromium browser
49+
workingDirectory: tests/Umbraco.Tests.AcceptanceTest
50+
51+
# Test
52+
- pwsh: ${{ parameters.testCommand }}
53+
displayName: Run Playwright tests
54+
continueOnError: true
55+
workingDirectory: tests/Umbraco.Tests.AcceptanceTest
56+
env:
57+
CI: true
58+
CommitId: $(Build.SourceVersion)
59+
AgentOs: $(Agent.OS)
60+
AZUREADB2CTESTUSEREMAIL: ${{ parameters.AZUREB2CTESTUSEREMAIL }}
61+
AZUREADB2CTESTUSERPASSWORD: ${{ parameters.AZUREB2CTESTUSERPASSWORD }}
62+
63+
# Stop application
64+
- bash: kill -15 $(AcceptanceTestProcessId)
65+
displayName: Stop application (Linux)
66+
condition: and(succeededOrFailed(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Linux'))
67+
68+
- pwsh: Stop-Process -Id $(AcceptanceTestProcessId)
69+
displayName: Stop application (Windows)
70+
condition: and(succeededOrFailed(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Windows_NT'))
71+
72+
- ${{ if eq(parameters.DatabaseType, 'SQLServer') }}:
73+
# Stop SQL Server
74+
- pwsh: docker stop mssql
75+
displayName: Stop SQL Server Docker image (Linux)
76+
condition: and(succeededOrFailed(), eq(variables['Agent.OS'], 'Linux'))
77+
78+
- pwsh: SqlLocalDB stop MSSQLLocalDB
79+
displayName: Stop SQL Server LocalDB (Windows)
80+
condition: and(succeededOrFailed(), eq(variables['Agent.OS'], 'Windows_NT'))
81+
82+
# Copy artifacts
83+
- pwsh: |
84+
if (Test-Path tests/Umbraco.Tests.AcceptanceTest/results/*) {
85+
Copy-Item tests/Umbraco.Tests.AcceptanceTest/results/* $(Build.ArtifactStagingDirectory) -Recurse
86+
}
87+
displayName: Copy Playwright results
88+
condition: succeededOrFailed()
89+
90+
# Publish
91+
- task: PublishPipelineArtifact@1
92+
displayName: Publish test artifacts
93+
condition: succeededOrFailed()
94+
inputs:
95+
targetPath: $(Build.ArtifactStagingDirectory)
96+
artifact: "Acceptance Test Results - $(Agent.JobName) - Attempt #$(System.JobAttempt)"
97+
98+
# Publish test results
99+
- task: PublishTestResults@2
100+
displayName: "Publish test results"
101+
condition: succeededOrFailed()
102+
inputs:
103+
testResultsFormat: 'JUnit'
104+
testResultsFiles: '*.xml'
105+
searchFolder: "tests/Umbraco.Tests.AcceptanceTest/results"
106+
testRunTitle: "$(Agent.JobName)"

‎build/nightly-E2E-setup-template.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
parameters:
2+
- name: nodeVersion
3+
type: string
4+
default: ''
5+
6+
- name: PlaywrightUserEmail
7+
type: string
8+
default: ''
9+
10+
- name: PlaywrightPassword
11+
type: string
12+
default: ''
13+
14+
- name: ASPNETCORE_URLS
15+
type: string
16+
default: ''
17+
18+
- name: npm_config_cache
19+
type: string
20+
default: ''
21+
22+
steps:
23+
- task: DownloadPipelineArtifact@2
24+
displayName: Download NuGet artifacts
25+
inputs:
26+
artifact: nupkg
27+
path: $(Agent.BuildDirectory)/app/nupkg
28+
29+
- task: NodeTool@0
30+
displayName: Use Node.js $(nodeVersion)
31+
inputs:
32+
versionSpec: $(nodeVersion)
33+
34+
- task: UseDotNet@2
35+
displayName: Use .NET SDK from global.json
36+
inputs:
37+
useGlobalJson: true
38+
39+
- pwsh: |
40+
"UMBRACO_USER_LOGIN=${{ parameters.PlaywrightUserEmail }}
41+
UMBRACO_USER_PASSWORD=${{ parameters.PlaywrightPassword }}
42+
URL=${{ parameters.ASPNETCORE_URLS }}
43+
STORAGE_STAGE_PATH=$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/playwright/.auth/user.json
44+
CONSOLE_ERRORS_PATH=$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/console-errors.json" | Out-File .env
45+
displayName: Generate .env
46+
workingDirectory: $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
47+
48+
# Cache and restore NPM packages
49+
- task: Cache@2
50+
displayName: Cache NPM packages
51+
inputs:
52+
key: 'npm_e2e | "$(Agent.OS)" | $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/package-lock.json'
53+
restoreKeys: |
54+
npm_e2e | "$(Agent.OS)"
55+
npm_e2e
56+
path: ${{ parameters.npm_config_cache }}
57+
58+
- script: npm ci --no-fund --no-audit --prefer-offline
59+
workingDirectory: $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
60+
displayName: Restore NPM packages
61+
62+
# Install Template
63+
- pwsh: |
64+
$cmsVersion = "$(Build.BuildNumber)" -replace "\+",".g"
65+
dotnet new nugetconfig
66+
dotnet nuget add source ./nupkg --name Local
67+
dotnet new install Umbraco.Templates::$cmsVersion
68+
dotnet new umbraco --name UmbracoProject --version $cmsVersion --exclude-gitignore --no-restore --no-update-check
69+
displayName: Install Template
70+
workingDirectory: $(Agent.BuildDirectory)/app

0 commit comments

Comments
 (0)