Provide Struct data type support for oracle plugin by vanshikaagupta22 · Pull Request #659 · data-integrations/database-plugins

vanshikaagupta22 · 2026-05-06T09:27:33Z

This change adds native support for resolving Oracle STRUCT types (Object Types) into CDAP RECORD schemas. By querying the ALL_TYPE_ATTRS metadata table, the schema builder dynamically processes complex structures with support for up to 4 levels of nesting. Internal attributes are first translated to standard SQL data types via a dedicated mapper before being converted into the final CDAP schema.
When reading the data, the implementation overrides the setField() function, providing a custom implementation to properly extract and map individual custom object attributes according to requirements.

Note: Users will require explicit EXECUTE privileges on the custom Oracle object types to successfully resolve the schema.

gemini-code-assist

Code Review

This pull request adds support for resolving Oracle STRUCT types into CDAP RECORD schemas by querying the ALL_TYPE_ATTRS metadata table. The implementation includes recursive resolution for nested structures and an updated mapping for primitive Oracle types. Feedback highlights several critical issues: the metadata query lacks an OWNER filter which could lead to incorrect schema resolution in multi-schema environments; fully qualified type names containing dots will cause IllegalArgumentException when creating CDAP records; and there are logic errors in the primitive type mapping, specifically regarding Oracle-specific type naming conventions and an invalid comparison between Java class names and SQL type strings.

gemini-code-assist · 2026-05-06T09:29:45Z

+    String sql = "SELECT * FROM ALL_TYPE_ATTRS WHERE TYPE_NAME = ? ORDER BY ATTR_NO";
+
+    try (PreparedStatement stmt = connection.prepareStatement(sql)) {
+      stmt.setString(1, typeName.substring(typeName.lastIndexOf('.') + 1));


The query to ALL_TYPE_ATTRS should include a filter on the OWNER column. Without it, if multiple schemas define a type with the same name, the query will return attributes for all of them, leading to an incorrect or corrupted schema.

Additionally, the typeName provided by the JDBC driver is often fully qualified (e.g., OWNER.TYPE_NAME). You should parse the owner from the typeName string if present, and fallback to the provided schemaName otherwise.

gemini-code-assist · 2026-05-06T09:29:45Z

+          if (attrSchema != null) {
+            fields.add(Schema.Field.of(attrName, attrSchema));
+          } else {
+            Schema nestedSchema = getStructSchema(connection, schemaName, attrTypeName);


When recursing for nested STRUCT types, the schemaName should be updated to the owner of the attribute's type to ensure the subsequent query filters correctly. This owner can be retrieved from the ATTR_TYPE_OWNER column in the ALL_TYPE_ATTRS result set.

gemini-code-assist · 2026-05-06T09:29:45Z

+              schemaName, typeName));
+    }
+
+    return Schema.recordOf(typeName, fields);


CDAP Schema.recordOf requires the record name to be a valid identifier (matching [A-Za-z_][A-Za-z0-9_]*). If typeName is fully qualified (e.g., MY_SCHEMA.MY_TYPE), this call will throw an IllegalArgumentException due to the dot. Consider using only the simple type name part for the record name.

String recordName = typeName.contains(".") ? typeName.substring(typeName.lastIndexOf('.') + 1) : typeName; return Schema.recordOf(recordName, fields);

gemini-code-assist · 2026-05-06T09:29:45Z

+      case "TIMESTAMP WITH TZ":
+        return isTimestampOldBehavior ? Schema.of(Schema.Type.STRING) : Schema.of(Schema.LogicalType.TIMESTAMP_MICROS);
+      case "TIMESTAMP WITH LTZ":
+        return getTimestampLtzSchema();
+      case "TIMESTAMP":
+        return Schema.of(Schema.LogicalType.DATETIME);
+      case "DATE" :
+        return Schema.of(Schema.LogicalType.DATE);
+      case "BINARY FLOAT":
+      case "FLOAT":
+        return Schema.of(Schema.Type.FLOAT);
+      case "BINARY DOUBLE":
+      case "DOUBLE":
+        return Schema.of(Schema.Type.DOUBLE);
+      case "BFILE":
+      case "RAW":
+      case "LONG RAW":
+        return Schema.of(Schema.Type.BYTES);
+      case "INTERVAL DAY TO SECOND":
+      case "INTERVAL YEAR TO MONTH":
+      case "VARCHAR2":
+      case "VARCHAR":
+      case "CHAR":
+      case "CLOB":
+      case "BLOB":
+      case "LONG":
+        return Schema.of(Schema.Type.STRING);
+      case "INTEGER":
+        return Schema.of(Schema.Type.INT);
+      case "NUMBER":
+      case "DECIMAL":
+        // FLOAT and REAL are returned as java.sql.Types.NUMERIC but with value that is a java.lang.Double
+        if (Double.class.getTypeName().equals(typeName)) {
+          return Schema.of(Schema.Type.DOUBLE);


There are two issues in mapPrimitiveOracleType:

Type Name Mismatches: The strings used in the switch (e.g., "TIMESTAMP WITH TZ", "BINARY FLOAT") likely do not match the values returned by Oracle in ALL_TYPE_ATTRS.ATTR_TYPE_NAME. For example, Oracle typically uses "TIMESTAMP WITH TIME ZONE", "BINARY_FLOAT", and "BINARY_DOUBLE". Please verify these against the Oracle metadata documentation.

Logic Error: At line 243, Double.class.getTypeName().equals(typeName) compares the Java class name string ("java.lang.Double") with the SQL type name (e.g., "NUMBER"), which will always be false. Since metadata is not available here to check the column class, this check should probably be removed or replaced with a check on the SQL type name if applicable.

# Conflicts: # oracle-plugin/src/main/java/io/cdap/plugin/oracle/OracleSourceSchemaReader.java # oracle-plugin/src/test/java/io/cdap/plugin/oracle/OracleSchemaReaderTest.java

vikasrathee-cs · 2026-06-19T12:25:32Z

+    return getBfileBytes(bfile, columnName);
+  }
+
+  public static byte[] getBfileBytes(Object bfile, String columnName) {


why is this static?

vikasrathee-cs · 2026-07-01T12:20:46Z

+    }
+    if (attrValue instanceof BigDecimal) {
+      BigDecimal bigDecimal = (BigDecimal) attrValue;
+      if (Schema.LogicalType.DECIMAL.equals(fieldSchema.getLogicalType())) {


create a separate method for handling bigdecimal

vikasrathee-cs · 2026-07-01T16:52:15Z

+   */
+  private static final Map<String, Integer> DATA_TYPE_MAP = new HashMap<>();
+  static {
+    DATA_TYPE_MAP.put("TIMESTAMP WITH LOCAL TZ", TIMESTAMP_LTZ);


does oracle stores this type as TIMESTAMP WITH LOCAL TIMEZONE also, verify once

vikasrathee-cs · 2026-07-01T16:56:53Z

+    StructuredRecord.Builder builder = StructuredRecord.builder(schema);
+
+    for (int index = 0; index < attributes.length; index++) {
+      Schema.Field field = fields.get(index);


Normal columns are resilient to schema/result-set ordering differences because they are matched by name (resultSet.findColumn(field.getName())).
STRUCT attributes rely on Struct.getAttributes() order exactly matching schema.getFields() order. If a user edits the output schema (drops, renames, or reorders an attribute — all legitimate in CDAP), this silently writes values into the wrong fields, or throws IndexOutOfBoundsException when counts differ.

vikasrathee-cs · 2026-07-01T16:59:21Z

+      recordBuilder.set(field.getName(), attrValue.toString());
+      return;
+    }
+    if ("oracle.sql.NCLOB".equals(attrClassName)) {


oracle.sql.NCLOB implements java.sql.Clob, so the instanceof Clob branch (getSubString) is evaluated before the "oracle.sql.NCLOB".equals(className) branch (toString()). The NCLOB-specific branch is unreachable.

vikasrathee-cs · 2026-07-01T17:03:17Z

    Assert.assertEquals(expectedSchemaFields.get(1).getSchema(), actualSchemaFields.get(1).getSchema());
  }

+  @Test


write test cases for all data types and multiple level of structs also.

gemini-code-assist Bot reviewed May 6, 2026

View reviewed changes

Provide Struct datat type support for oracle plugin

231c6ee

vanshikaagupta22 force-pushed the oracle-structsupport branch from a88c6b7 to 231c6ee Compare May 6, 2026 14:13

Fix data reading errors

fe03c7b

vanshikaagupta22 force-pushed the oracle-structsupport branch from e6834e3 to fe03c7b Compare May 21, 2026 12:23

Merge branch 'develop' into oracle-structsupport

5ab3cd8

# Conflicts: # oracle-plugin/src/main/java/io/cdap/plugin/oracle/OracleSourceSchemaReader.java # oracle-plugin/src/test/java/io/cdap/plugin/oracle/OracleSchemaReaderTest.java

vishwasvaidya-cloudsufi force-pushed the oracle-structsupport branch 2 times, most recently from a11f5cc to af67d36 Compare June 15, 2026 10:01

Improve STRUCT support and expand data type mappings

3bfd8ba

vishwasvaidya-cloudsufi force-pushed the oracle-structsupport branch from af67d36 to 3bfd8ba Compare June 17, 2026 06:11

vanshikaagupta22 force-pushed the oracle-structsupport branch from f4ab512 to 47834bb Compare June 19, 2026 10:27

Updated schema mapping logic

219b85e

vanshikaagupta22 force-pushed the oracle-structsupport branch from 47834bb to 219b85e Compare June 19, 2026 10:53

vikasrathee-cs reviewed Jun 19, 2026

View reviewed changes

vishwasvaidya-cloudsufi and others added 2 commits June 25, 2026 15:38

Refactor Oracle record parsing and remove old struct converters

a33a1d3

Fixed data type mapping

d8f92c3

vikasrathee-cs added the build label Jun 29, 2026

vikasrathee-cs reviewed Jul 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Provide Struct data type support for oracle plugin#659

Provide Struct data type support for oracle plugin#659
vanshikaagupta22 wants to merge 7 commits into
data-integrations:developfrom
cloudsufi:oracle-structsupport

vanshikaagupta22 commented May 6, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 6, 2026

Uh oh!

gemini-code-assist Bot May 6, 2026

Uh oh!

gemini-code-assist Bot May 6, 2026

Uh oh!

gemini-code-assist Bot May 6, 2026

Uh oh!

vikasrathee-cs Jun 19, 2026

Uh oh!

vikasrathee-cs Jul 1, 2026

Uh oh!

vikasrathee-cs Jul 1, 2026

Uh oh!

vikasrathee-cs Jul 1, 2026

Uh oh!

vikasrathee-cs Jul 1, 2026

Uh oh!

vikasrathee-cs Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

vanshikaagupta22 commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

vikasrathee-cs Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vanshikaagupta22 commented May 6, 2026 •

edited

Loading