feat: add OpenAI tool calling for Responses, Chat, and Prompt

[Rana Singh (ranadeepsingh)]

What this changes

• Adds function/tool calling to OpenAIResponses, OpenAIChatCompletion, and both OpenAI modes of OpenAIPrompt.
• Supports scalar and per-row tools, tool_choice, and parallel_tool_calls, plus Responses-only max_tool_calls.
• Converts one Spark-facing function definition to each endpoint's wire contract:
  • Chat Completions: nested tools[].function
  • Responses: flat function tools
• Projects Chat and Responses calls into one structured toolCallsCol DataFrame contract.
• Supports typed Responses function_call_output continuations and Chat assistant tool_calls plus role="tool" continuation messages.
• Adds modern Responses request fields, Python list/dict ergonomics, generated column helpers, offline/local-HTTP integration coverage, and DataFrame-oriented notebooks.

Endpoint contracts

Concern	Chat Completions	Responses
Assistant call	choices[].message.tool_calls[]	output[] item with type: "function_call"
Correlation key	tool_calls[].id	function_call.call_id
Tool result	role: "tool" + tool_call_id	type: "function_call_output" + call_id
Arguments	JSON-encoded string	JSON-encoded string
Stored continuation	Replay the message history	previous_response_id or conversation
Stateless continuation	Replay messages	Replay the original input and every prior output item, including encrypted reasoning items

SynapseML transports definitions, calls, and results; it does not execute application functions. User code must validate the model-generated name/arguments, perform authorized work, and return one result for every call ID.

Rebase and hardening

The original 13 commits are rebased onto master at 3498243d0b5534117eb24c0c3cebde3eb1c1a963. The final review head is 6b3b7a026e8b3d4362ca628067b3b74ead51f1cf.

The rebased implementation additionally:

• preserves target-branch multimodal message validation while retaining nested Chat tool calls;
• supports Responses continuation-only rows without requiring messagesCol;
• silently skips null/empty Chat message arrays without an output, error, or HTTP request;
• keeps request serialization null-safe and isolates malformed row-bound values in errorCol;
• validates strict function schemas recursively, including additionalProperties: false, complete required sets, and nested/composed object schemas;
• warns on Azure when strict schemas are combined with parallel calls, while preserving the caller's payload;
• rejects collisions among public input/output columns before execution;
• declares Responses continuation array elements, call_id, and output non-nullable while retaining runtime guards;
• makes Chat reject the Responses-only replayItemsColumn helper with a clear UnsupportedOperationException;
• keeps the generated Chat Python surface limited to supported helpers;
• validates Chat content before recursively encoding the complete row so malformed short rows produce stable structural errors; and
• replays the minimal missing OpenAI multimodal baseline for Spark 4.1 while preserving that branch's Scala 2.13 collection adaptations.

Strict schemas and Azure

For strict: true, every object schema must set additionalProperties: false, every defined property must appear in required, and required cannot name undefined properties. Optional business values remain required keys with nullable types such as ["string", "null"].

Current Azure guidance requires parallel_tool_calls: false with strict Structured Outputs. The notebooks set it explicitly. SynapseML warns rather than silently mutating or rejecting the request because endpoint and proxy capabilities can differ.

Compatibility and deliberate limits

• Existing text-first OpenAIPrompt behavior and the default Chat Completions mode remain unchanged.
• Public JVM response case-class arities remain unchanged; widened wire parsing uses internal V2 schemas.
• gpt-5-mini and gpt-5.1 currently document Chat Completions, Responses, function calling, and Structured Outputs support. SynapseML does not infer deployment capability.
• The typed Responses continuation column accepts string outputs. Newer image/file output arrays require raw input items or a future typed extension.
• Provider built-in/custom tools can pass through, but typed projection and automatic orchestration are function-call focused.
• Spark HTTP work is at least once. Materialize paid turns and make side effects idempotent by call ID.

Validation

Final-head local validation:

• 60 tests passed across the six offline OpenAI tool-calling suites, including Chat and Responses local-HTTP integrations.
• 25 tests passed across the two multimodal suites that reproduced the Azure OpenAI shard failures.
• Cognitive main/test Scala compilation completed under Java 11 / Spark 3.5.
• Cognitive main and test Scalastyle completed with zero errors, warnings, or infos.
• The exact Spark 4.1 release replay completed full sbt test:compile under Java 17 / Scala 2.13.
• All 72 release-pipeline configuration tests passed.
• Black left all 202 checked Python files unchanged.
• Exact payload tests cover initial calls, stored/stateless continuations, row-isolated validation errors, shared projections, Prompt delegation, persistence, and generated Python surface rules.

The rebased series also completed sbt codegen, the focused generated-wrapper Python suite (5/5), and notebook JSON/Python-cell validation before the final compatibility follow-up. No credentialed OpenAI request was run locally for this final head. Exact-head Azure validation build 232986628 passed all 77 reported checks, including OpenAI, Spark 4.1 release compatibility, Search1, and LightGBM4.

References

• OpenAI function calling
• OpenAI conversation state
• OpenAI reasoning items
• OpenAI gpt-5-mini
• OpenAI gpt-5.1
• Azure OpenAI function calling
• Azure OpenAI Responses
• Azure Structured Outputs

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[github-actions]

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

This PR extends SynapseML’s OpenAI integrations to support the OpenAI Responses API tool/function-calling workflow end-to-end (tool declarations, structured tool calls, and typed continuation outputs), while expanding support for newer Responses parameters and improving schema/DTO compatibility.

Changes:

• Adds tool normalization/validation utilities and Spark column helpers for tool calls, replay items, and typed function_call_output continuations.
• Extends OpenAIResponses and OpenAIPrompt to support scalar + per-row tool configuration, modern Responses fields, and compatibility-preserving response parsing (via an internal V2 schema).
• Adds Scala + Python tests (offline, integration, and opt-in live) plus documentation notebooks illustrating safe, explicit DataFrame workflows.

Show a summary per file

File	Description
docs/Explore Algorithms/OpenAI/Quickstart - OpenAI Responses Tool Calling.ipynb	New notebook documenting a two-turn tool-calling + continuation Spark workflow, with safety/idempotency guidance.
docs/Explore Algorithms/OpenAI/Quickstart - Multimodal OpenAI Prompter with Responses API.ipynb	Adds a “next steps” section pointing users to the tool-calling workflow.
core/src/test/scala/com/microsoft/azure/synapse/ml/param/VerifyUntypedArrayParam.scala	Expands test coverage for widened AnyJsonFormat encoding/decoding (null/Long/Float/JsValue).
core/src/main/scala/com/microsoft/azure/synapse/ml/param/UntypedArrayParam.scala	Widens AnyJsonFormat to support JsValue, null, Long, and Float serialization, and JsNull reads.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/ToolTestFixtures.scala	Introduces shared fixtures for tool JSON and representative Responses payloads.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtilsSuite.scala	Adds unit tests for tool parsing/normalization, tool choice validation, and schema contracts.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolsLiveSuite.scala	Adds credential-gated live smoke tests for real tool calls and continuations.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesToolsSuite.scala	Adds comprehensive offline tests for serialization, per-row tool params, continuation ordering, and response parsing behavior.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesToolsIntegrationSuite.scala	Adds local HTTP stub integration tests to verify executor behavior without external dependencies.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesSuite.scala	Updates tests to use the internal V2 Responses schema.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala	Adds Prompt-level tests ensuring tool/modern params forwarding and explicit Responses-only validation.
cognitive/src/test/python/synapsemltest/services/openai/test_OpenAIResponsesTools.py	Adds Python tests for tool setter ergonomics, column helpers, and save/load behavior.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtils.scala	Adds pure tool/toolChoice/input-items parsing + normalization + validation utilities for Responses tool payloads.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolPythonOverrides.scala	Adds generated Python convenience methods for tool configuration and column helpers.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolParams.scala	Adds shared tool params + Prompt opt-in structured outputs (toolCallsCol, responseStructCol).
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala	Adds Spark column helpers for extracting tool calls and deterministic replay items from Responses structs.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAISchemas.scala	Adds an internal V2 Responses schema to widen parsing while keeping public DTOs stable.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesModernParams.scala	Adds modern Responses parameters and merging logic (including nested reasoning extras).
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala	Implements tool payload support, continuation inputs, V2 schema parsing, tool call projection, and row-level validation.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPrompt.scala	Wires tool + modern Responses params through Prompt, adds Responses-only validation, and adds optional structured outputs.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (2)

cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala:139

• The standalone spark expression is a no-op and can be removed (it doesn’t affect the test and can trip style checks for unused expressions).

    spark
    val prompt = new OpenAIPrompt()

cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala:161

• The standalone spark expression is a no-op and can be removed (it doesn’t affect the test and can trip style checks for unused expressions).

    spark
    val prompt = configuredPrompt

• Files reviewed: 20/20 changed files
• Comments generated: 2
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:161

• prepareEntity calls .toSeq on the result of r.getAs[Seq[Row]](getMessagesCol). If messagesCol exists but the row value is null (common for continuation-only rows that still carry a nullable messages column), this will throw a NPE before the request is built. Treat null messages as an empty sequence so stored continuations can run without requiring non-null messages per row.

  override protected[openai] def prepareEntity: Row => Option[AbstractHttpEntity] = {
    r =>
      lazy val optionalParams: Map[String, Any] = getOptionalParams(r)
      val messages = r.getAs[scala.collection.Seq[Row]](getMessagesCol).toSeq
      Some(getStringEntity(messages, optionalParams))
  }

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala:37

• FunctionCallOutputStructType does not set containsNull = false, but toFunctionCallOutputs assumes every array element is a non-null Row and will throw if a null element is present. Mark the array as non-nullable and fail fast with a clear error if a null element still appears.

  val FunctionCallOutputStructType: ArrayType = ArrayType(StructType(Seq(
    StructField("call_id", StringType),
    StructField("output", StringType),
    StructField("status", StringType)
  )))

• Files reviewed: 23/23 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala:30

• FunctionCallOutputStructType currently allows null array elements and marks call_id/output/status as nullable, but toFunctionCallOutputs later requires non-blank call_id and non-null output (and will NPE if an array element itself is null). Tightening the schema to containsNull=false and making call_id/output non-nullable helps catch invalid continuation rows earlier (schema/analysis) and keeps the contract consistent with ToolCallStructType.

  val FunctionCallOutputStructType: ArrayType = ArrayType(StructType(Seq(
    StructField("call_id", StringType),
    StructField("output", StringType),
    StructField("status", StringType)
  )))

• Files reviewed: 23/23 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

Review details

• Files reviewed: 29/29 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:28

• ResponseFormat uses the misspelled identifier paylodName. To improve readability without forcing updates outside this diff hunk, rename the constructor field to payloadName and add a backward-compatible paylodName alias so existing references still compile.

object OpenAIResponseFormat extends Enumeration {
  case class ResponseFormat(paylodName: String) extends super.Val(paylodName)

• Files reviewed: 29/29 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (5)

Previously missed (5) — in code that hasn't changed since the last review.

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtils.scala:433

• parseInputItems accepts either a JSON array or a single JSON object (returning Vector(obj)), but the thrown error message only mentions arrays. This is misleading for users debugging malformed row-bound values.

  def parseInputItems(json: String): Vector[JsValue] =
    Option(json).map(_.trim).filter(_.nonEmpty).map { text =>
      Try(text.parseJson).toOption match {
        case Some(JsArray(items)) => items
        case Some(obj: JsObject) => Vector(obj)
        case _ =>
          throw new IllegalArgumentException(
            "inputItemsCol must contain a JSON array of Responses input items")
      }

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolParams.scala:348

• validatePromptToolOutputColumns checks for existing columns using schema.fieldNames.contains, which is case-sensitive. In Spark's default case-insensitive mode, this can allow toolCallsCol/responseStructCol to collide with an existing column that differs only by case, leading to overwrite or analysis errors at runtime.

    val requested = Seq(get(toolCallsCol), get(responseStructCol)).flatten
    requested.foreach { columnName =>
      require(
        !schema.fieldNames.contains(columnName),
        s"Column '$columnName' already exists in the input DataFrame")
    }

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:631

• The toolCallsCol collision check uses schema.fieldNames.contains, which is case-sensitive. With Spark's default case-insensitive resolver, this can miss an existing column that differs only by case and allow overwriting it.

    get(toolCallsCol).foreach { columnName =>
      require(
        !schema.fieldNames.contains(columnName),
        s"Column '$columnName' already exists in the input DataFrame")
    }

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:701

• requireFieldShape looks up a column with schema.find(_.name == columnName), which is case-sensitive. In Spark's default case-insensitive mode this can incorrectly report that a configured column doesn't exist (or fail to validate the correct one).

  private def requireFieldShape(
      schema: StructType,
      columnName: String,
      expectedType: DataType): Unit = {
    val actualType = schema.find(_.name == columnName).map(_.dataType).getOrElse {
      throw new IllegalArgumentException(
        s"Column '$columnName' was not found in the input DataFrame")
    }

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala:668

• validateChatToolSetup checks for existing toolCallsCol using schema.fieldNames.contains, which is case-sensitive. In Spark's default case-insensitive mode, this can miss collisions with existing columns that differ only by case and lead to overwriting/analysis errors.

  private def validateChatToolSetup(schema: StructType): Unit = {
    get(toolCallsCol).foreach { columnName =>
      require(
        !schema.fieldNames.contains(columnName),
        s"Column '$columnName' already exists in the input DataFrame")
    }

• Files reviewed: 29/29 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

• Files reviewed: 30/30 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:704

• The Azure strict-schema warning suggests only the scalar setter (setParallelToolCalls(false)), but parallelToolCalls can also be column-bound. With setParallelToolCallsCol(...), this message is misleading (and may warn even when all rows set false). Adjust the wording to mention both scalar and per-row configuration so users can fix the issue correctly.

  private def warnOnAzureStrictParallel(isAzure: Boolean): Unit = {
    if (isAzure && strictToolsNeedSequentialCalls) {
      logWarning(
        "Azure OpenAI strict function schemas require parallel_tool_calls=false; " +
          "call setParallelToolCalls(false) to avoid HTTP 400")
    }

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala:675

• This Azure strict-schema warning text only mentions setParallelToolCalls(false), but parallelToolCalls can be set per-row via setParallelToolCallsCol(...). Updating the message avoids pushing callers toward the wrong configuration approach when they intentionally use row-bound params.

    val configuredUrl = get(url).orElse(getDefault(url)).map(_.toLowerCase)
    if (configuredUrl.exists(_.contains("azure.com")) && strictToolsNeedSequentialCalls) {
      logWarning(
        "Azure OpenAI strict function schemas require parallel_tool_calls=false; " +
          "call setParallelToolCalls(false) to avoid HTTP 400")
    }

• Files reviewed: 32/32 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 91.22316% with 94 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.57%. Comparing base (3498243) to head (6b3b7a0).

Files with missing lines	Patch %	Lines
...e/synapse/ml/services/openai/OpenAIToolUtils.scala	81.43%	44 Missing ⚠️
.../services/openai/OpenAIResponsesMessageUtils.scala	84.21%	21 Missing ⚠️
.../synapse/ml/services/openai/OpenAIToolParams.scala	92.53%	10 Missing ⚠️
...e/synapse/ml/services/openai/OpenAIResponses.scala	96.37%	9 Missing ⚠️
.../services/openai/OpenAIResponsesModernParams.scala	96.06%	5 Missing ⚠️
...apse/ml/services/openai/OpenAIChatCompletion.scala	95.65%	4 Missing ⚠️
...synapse/ml/services/openai/OpenAIToolColumns.scala	98.38%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2626      +/-   ##
==========================================
+ Coverage   87.33%   87.57%   +0.24%     
==========================================
  Files         338      344       +6     
  Lines       20234    21132     +898     
  Branches     2059     2192     +133     
==========================================
+ Hits        17671    18506     +835     
- Misses       2563     2626      +63     

Files with missing lines	Coverage Δ
...zure/synapse/ml/services/openai/OpenAIPrompt.scala	95.54% <100.00%> (+1.59%)	⬆️
...ure/synapse/ml/services/openai/OpenAISchemas.scala	100.00% <ø> (ø)
...ml/services/openai/OpenAIToolPythonOverrides.scala	100.00% <100.00%> (ø)
...oft/azure/synapse/ml/param/UntypedArrayParam.scala	83.33% <100.00%> (+5.91%)	⬆️
...synapse/ml/services/openai/OpenAIToolColumns.scala	98.38% <98.38%> (ø)
...apse/ml/services/openai/OpenAIChatCompletion.scala	89.87% <95.65%> (+1.39%)	⬆️
.../services/openai/OpenAIResponsesModernParams.scala	96.06% <96.06%> (ø)
...e/synapse/ml/services/openai/OpenAIResponses.scala	94.03% <96.37%> (+6.02%)	⬆️
.../synapse/ml/services/openai/OpenAIToolParams.scala	92.53% <92.53%> (ø)
.../services/openai/OpenAIResponsesMessageUtils.scala	84.21% <84.21%> (ø)
... and 1 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.