test: add Fabric OpenAIPrompt end-to-end gate

[Rana Singh (ranadeepsingh)]

Related Issues/PRs

Motivated by the real-runtime validation gap exposed in #2662.

What changes are proposed in this pull request?

Add a reusable, manifest-driven fabric-spark-cli path that runs exact
SynapseML build artifacts on a managed Fabric Spark runtime and retains
machine-readable evidence.

• Add batch and platform-notebook execution profiles with unique scratch
  lakehouses/notebooks and exact cleanup in finally.
• Record the source commit, jar SHA-256 hashes, runtime/application metadata,
  class-source provenance, diagnostics, JUnit, CLI output, and executed
  notebook.
• Add runtime smoke, jar provenance, LightGBM streaming, and
  OpenAIPrompt scenarios.
• Exercise OpenAIPrompt with AI Functions-inspired generation, sentiment,
  summarization, translation, structured extraction, null propagation, usage,
  and error-column cases.
• Use Fabric's implicit LLM authentication from a platform notebook; do not
  pass an endpoint, OpenAI key, subscription key, or AAD token.
• Extend FabricE2E to build the exact core/cognitive package jars, run the
  notebook scenario for master and spark3.5, skip untrusted fork PRs, and
  publish JUnit plus retained evidence.
• Add a repository skill describing repeatable local and PR workflows.

Platform-notebook execution is intentional: direct Fabric batch submission
does not carry the workload-operation context required by the implicit Fabric
LLM endpoint.

Managed proof completed with the exact commit jars:

• Fabric Spark runtime: 3.5.5.5.4.20260807.1
• CLI: fabric-spark-cli 0.1.20260807.5
• Scenario: openai-prompt-ai-functions
• Result: passed, including exact core/cognitive class provenance
• Cleanup: notebook and lakehouse both exited 0

The managed proof used developer authentication. This PR's FabricE2E run is
the remaining proof of the SynapseML Build service connection's workspace
permissions.

How is this patch tested?

• I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

• python -m pytest -q tools/ci/tests/test_fabric_e2e.py tools/ci/tests/test_pipeline_yaml.py

• sbt scalastyle test:scalastyle with the repository-supported JDK

• black --check --extend-exclude 'docs/' over tracked and new Python sources

• Managed Fabric execution of the checked-in OpenAIPrompt scenario with exact
  core/cognitive jars

Does this PR change any dependencies?

• No. You can skip this section.
• Yes. Make sure the dependencies are resolved correctly, and list changes here.

The pipeline installs pinned fabric-spark-cli==0.1.20260807.5 from the
existing authenticated SynapseMaven feed. No published SynapseML dependency or
dependency manifest changes.

Does this PR add a new feature? If so, have you added samples on website?

• No. You can skip this section.
• Yes. Make sure you have added samples following below steps.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[github-actions]

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[Copilot]

Pull request overview

This PR adds a manifest-driven Fabric end-to-end (E2E) runner (tools/fabric_e2e/run.py) that executes checked-in scenarios on a managed Microsoft Fabric Spark runtime via fabric-spark-cli, producing machine-readable evidence (JUnit + JSON + retained logs/notebooks). It also extends CI (pipeline.yaml) to gate an OpenAIPrompt notebook-based scenario on trusted builds and publishes results/evidence.

Changes:

• Added a reusable Fabric E2E runner with scenario manifest validation, jar hashing/provenance checks, structured diagnostics, JUnit output, and deterministic cleanup of scratch resources.
• Added multiple Fabric E2E scenarios (runtime smoke, jar provenance, LightGBM streaming, OpenAIPrompt AI Functions-inspired checks) plus unit tests validating runner/manifest/pipeline wiring.
• Updated Azure Pipelines FabricE2E job to skip untrusted fork PRs, add an OpenAIPrompt gate step, and publish OpenAIPrompt JUnit + evidence artifacts.

Show a summary per file

File	Description
tools/fabric_e2e/scenarios/runtime_smoke.py	Minimal distributed Spark action + evidence marker for Fabric access proof.
tools/fabric_e2e/scenarios/jar_provenance.py	Validates that a specified JVM class loads from a supplied jar and records evidence.
tools/fabric_e2e/scenarios/lightgbm_streaming.py	Exercises repeated LightGBM streaming fits and captures native/library diagnostics on Fabric.
tools/fabric_e2e/scenarios/openai_prompt_ai_functions.py	Notebook-executed OpenAIPrompt scenario validating implicit Fabric auth and structured output behaviors.
tools/fabric_e2e/scenarios.json	Scenario manifest defining scripts, minimum jars, execution mode, and default args/conf.
tools/fabric_e2e/run.py	Core runner: manifest-driven execution, evidence capture, jar hashing, log parsing, and cleanup.
tools/ci/tests/test_fabric_e2e.py	Unit tests for runner command construction, manifest loading, marker parsing, and notebook generation.
tools/ci/tests/test_pipeline_yaml.py	CI assertions ensuring the Fabric OpenAIPrompt gate is wired as intended.
pipeline.yaml	CI updates: fork-skipping FabricE2E, OpenAIPrompt Fabric gate, publishing results/evidence, and new branch gating variable.
.github/skills/fabric-e2e/SKILL.md	New repo skill documenting repeatable Fabric E2E workflows and constraints.
.github/skills/fabric-e2e/references/scenarios.md	Reference doc describing scenarios, required jars, overrides, and outputs.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 11/11 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

Superseded by #2669. The replacement uses the same commit from a trusted upstream branch so the credential-bearing FabricE2E job can execute instead of being correctly skipped for a fork PR.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.56%. Comparing base (9f152c2) to head (018eb5b).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2668      +/-   ##
==========================================
- Coverage   86.64%   86.56%   -0.08%     
==========================================
  Files         337      337              
  Lines       19556    19556              
  Branches     1928     1928              
==========================================
- Hits        16944    16929      -15     
- Misses       2612     2627      +15     

see 11 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.