Skip to content

Scope PR test runs to affected apps and cover .github in fullBuildPatterns - #10306

Draft
Alexander Holstrup (aholstrup1) wants to merge 2 commits into
mainfrom
aholstrup1-super-disco
Draft

Scope PR test runs to affected apps and cover .github in fullBuildPatterns#10306
Alexander Holstrup (aholstrup1) wants to merge 2 commits into
mainfrom
aholstrup1-super-disco

Conversation

@aholstrup1

@aholstrup1 Alexander Holstrup (aholstrup1) commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What & why

BCApps currently runs the full test suite (~316 test apps) on every PR, even when a change touches a single app. The build-optimization logic to scope test runs to only the affected apps already existed in BuildOptimization.psm1 (with unit tests), but nothing in the pipeline called it. This wires it in and fixes a related gap in fullBuildPatterns.

Two changes:

  • Wire up per-app test optimization. Invoke-PerProjectTestRun now filters the per-project test-app list through Test-ShouldSkipTestApp, so a PR only runs tests for apps affected by its changed files plus their dependents. Filtering happens before dispatch, so skipped apps never occupy a tenant. This is correctness-preserving: a full run is still forced for fullBuildPatterns matches, unmapped files under src/, non-CI runs, and workflow_dispatch, so the failure mode is always "run too much," never "skip something it shouldn't."

  • Cover .github/* in fullBuildPatterns. Previously only two specific workflow files were listed. A change to .github/AL-Go-Settings.json (which defines artifact version, analyzers, target country, and the patterns themselves) mapped to no app, produced an empty affected set, and would have skipped every test. Broadening to .github/* closes that hole and subsumes the two explicit workflow entries.

On a sample of 10 recent open PRs, 6 dropped from 316 test apps to 4 or fewer; the rest correctly stayed full (base-layer or infra changes).

Linked work

Fixes #

How I validated this

  • I read the full diff and it contains only changes I intended.
  • I built the affected app(s) locally with no new analyzer warnings.
  • I ran the change in Business Central and confirmed it behaves as expected.
  • I added or updated tests for the new behavior, or explained below why none are needed.

What I tested and the outcome

  • Ran the existing BuildOptimization.Test.ps1 Pester suite: 36/37 pass. The one failure (Get-AffectedApps ... Expected 51, but got 56) is a pre-existing hardcoded-count assertion that also fails on main with these changes stashed; it is dependency-graph drift from apps added since the count was written, unrelated to this change.
  • Verified .github/AL-Go-Settings.json still parses and ParallelTestExecution.psm1 imports cleanly.
  • Replayed the affected-app computation against 10 open PRs to confirm scoping behaves as expected (single-app PRs scope down to 1-4 test apps; base-layer/infra PRs force a full run via fullBuildPatterns).
  • No new behavior-specific tests added: the filtering reuses the already-tested Test-ShouldSkipTestApp, and this change is wiring plus a settings entry.

Risk & compatibility

  • The optimization only narrows test runs on PR builds; it is a no-op outside CI and whenever a full run is required, so it cannot cause fewer tests to run than intended in those cases.
  • Scoping accuracy depends on app.json dependency declarations being correct (already how build order is resolved). A product app with no linked test app resolves to zero affected test apps; that is correct when the app genuinely has no tests, but a follow-up guard could warn when a changed product app has no associated test app.
  • Broadening to .github/* means build-irrelevant edits (e.g. PULL_REQUEST_TEMPLATE.md, CODEOWNERS) now force a full run. This is intentional and consistent with the conservative, correctness-first intent of the list.
  • Pre-existing stale test count (51 vs 56) is worth fixing separately so the suite goes green.

Filter the per-project test-app list through Test-ShouldSkipTestApp so PR
builds only run tests for apps affected by the changed files (plus their
dependents). The guard is correctness-preserving: a full run is still
forced for fullBuildPatterns matches, unmapped src files, non-CI, and
workflow_dispatch.

Also broaden the fullBuildPatterns entry to '.github/*' so changes to
AL-Go-Settings.json and other .github config force a full run instead of
silently skipping all tests. This subsumes the two explicit workflow
entries.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 08f8114e-ab0d-4d8b-8941-f65880a64825
@github-actions github-actions Bot added Build: Automation Workflows and other setup in .github folder Build: scripts & configs Build scripts and configuration files Other GitHub request for other area than SCM, Finance or Integration Ownership: Needs Review Ownership is Other, low confidence, or needs manual correction labels Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Could not find a linked ADO work item. Please link one by using the pattern 'AB#' followed by the relevant work item number. You may use the 'Fixes' keyword to automatically resolve the work item when the pull request is merged. E.g. 'Fixes AB#1234'

The best-effort base-commit fetch and the git diff in Get-ChangedFilesForCI
suppressed stderr, so when change detection fell back to a full test run we
couldn't tell why. Capture and log the fetch exit code/output and include
the git diff stderr in the failure message to diagnose why the base SHA is
unavailable on shallow self-hosted runners.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 08f8114e-ab0d-4d8b-8941-f65880a64825
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build: Automation Workflows and other setup in .github folder Build: scripts & configs Build scripts and configuration files Other GitHub request for other area than SCM, Finance or Integration Ownership: Needs Review Ownership is Other, low confidence, or needs manual correction

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant