Remove `UNSUPPORTED` labels from tests that are supported by bogner · Pull Request #1487 · llvm/offload-test-suite

bogner · 2026-08-28T00:42:14Z

These all should be XFAILs, but they were marked as unsupported. Many of these pass now, so in those cases I've just removed them.

There are two more interesting cases:

countbits.16 had a mistake in the test and is failing for clang. I filed [DirectX][Scalarizer] Failure to legalizer cast via vector for 16-bit countbits llvm-project#219327 to track the clang failure.
SwitchBranchAttr and SwitchFlattenAttr had the attribute in the wrong place and weren't actually testing what they said they were.

farzonl · 2026-08-28T16:52:48Z

-# Vulkan KosmicKrisp and MoltenVK the output is always False.
-# UNSUPPORTED: Metal || KosmicKrisp || MoltenVK
+# Note: For Vulkan KosmicKrisp and MoltenVK the output is always False.
+# UNSUPPORTED: KosmicKrisp || MoltenVK


Should we make this an XFAIL? If Metal is supported, then this would imply vulkan om Mac might eventually start working?

Maybe. I'm not set up to test either of these and we don't have them in CI, so I was uncomfortable changing them.

I removed all the KosmicKrisp || MoltenVK for the boolean matrix test and ran on both vulkan implementation and this issue appears to be resolved.

Full Results:

KosmicKrisp Clang

******************** Failed Tests (11): OffloadTest-clang-vk :: Bugs/UAV-Sequental-Consistency.yaml OffloadTest-clang-vk :: Feature/HLSLLib/select.32.test OffloadTest-clang-vk :: Feature/HLSLLib/select.fp16.test OffloadTest-clang-vk :: Feature/MaximalReconvergence/loop_peeling.test OffloadTest-clang-vk :: Feature/ResourcesInStructs/res-of-matrix-in-struct.test OffloadTest-clang-vk :: Feature/Textures/Array.CalculateLevelOfDetail.test OffloadTest-clang-vk :: Feature/Textures/CalculateLevelOfDetail.test OffloadTest-clang-vk :: Feature/TypedBuffer/64bit-scalar.test OffloadTest-clang-vk :: Tools/Offloader/BufferFormats.test OffloadTest-clang-vk :: WaveOps/WaveActiveBitXor.int.test OffloadTest-clang-vk :: WaveOps/WaveActiveBitXor.int64.test ******************** Unexpectedly Passed Tests (2): OffloadTest-clang-vk :: Feature/HLSLLib/abs.long-vector.32.test OffloadTest-clang-vk :: WaveOps/GroupMemoryBarrierWithGroupSync.test

KosmicKrisp DXC

******************** Failed Tests (20): OffloadTest-vk :: Bugs/UAV-Sequental-Consistency.yaml OffloadTest-vk :: Feature/MaximalReconvergence/loop_peeling.test OffloadTest-vk :: Feature/ResourceArrays/overflow-unbounded-array.test OffloadTest-vk :: Feature/Semantics/CullDistance.test OffloadTest-vk :: Feature/Semantics/DomainSystemValues.test OffloadTest-vk :: Feature/Semantics/HullSystemValues.test OffloadTest-vk :: Feature/Textures/Array.CalculateLevelOfDetail.test OffloadTest-vk :: Feature/Textures/CalculateLevelOfDetail.test OffloadTest-vk :: Feature/TypedBuffer/64bit-scalar.test OffloadTest-vk :: Feature/Vk.SampledTextures/Vk.SampledTexture2D/Vk.SampledTexture2D.CalculateLevelOfDetail.test.yaml OffloadTest-vk :: Graphics/IsolineDomainTessellation.test OffloadTest-vk :: Graphics/QuadDomainTessellation.test OffloadTest-vk :: Graphics/gs_point_to_quad.test OffloadTest-vk :: Graphics/gs_selective_output.test OffloadTest-vk :: Tools/Offloader/BufferFormats.test OffloadTest-vk :: WaveOps/WaveActiveBitAnd.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitOr.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitXor.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitXor.int.test OffloadTest-vk :: WaveOps/WaveActiveBitXor.int64.test

MoltenVK Clang

******************** Failed Tests (20): OffloadTest-clang-vk :: Bugs/UAV-Sequental-Consistency.yaml OffloadTest-clang-vk :: Feature/HLSLLib/select.32.test OffloadTest-clang-vk :: Feature/HLSLLib/select.fp16.test OffloadTest-clang-vk :: Feature/MaximalReconvergence/loop_peeling.test OffloadTest-clang-vk :: Feature/ResourcesInStructs/res-of-matrix-in-struct.test OffloadTest-clang-vk :: Feature/Textures/Array.GatherCmp.test OffloadTest-clang-vk :: Feature/Textures/Array.SampleBias.test OffloadTest-clang-vk :: Feature/Textures/Array.SampleCmp.test OffloadTest-clang-vk :: Feature/Textures/GatherCmp.test OffloadTest-clang-vk :: Feature/Textures/Sample.test OffloadTest-clang-vk :: Feature/Textures/SampleBias.test OffloadTest-clang-vk :: Feature/Textures/SampleCmp.test OffloadTest-clang-vk :: Feature/TypedBuffer/64bit-scalar.test OffloadTest-clang-vk :: Tools/Offloader/BufferFormats.test OffloadTest-clang-vk :: WaveOps/QuadReadAcrossDiagonal.int64.test OffloadTest-clang-vk :: WaveOps/QuadReadAcrossX.int64.test OffloadTest-clang-vk :: WaveOps/QuadReadAcrossY.int64.test OffloadTest-clang-vk :: WaveOps/WaveActiveBitAnd.int64.test OffloadTest-clang-vk :: WaveOps/WaveActiveBitOr.int64.test OffloadTest-clang-vk :: WaveOps/WaveActiveBitXor.int64.test ******************** Unexpectedly Passed Tests (1): OffloadTest-clang-vk :: Feature/HLSLLib/abs.long-vector.32.test

MoltenVK DXC

Failed Tests (24): OffloadTest-vk :: Bugs/UAV-Sequental-Consistency.yaml OffloadTest-vk :: Feature/CBuffer/arrays-16bit.test OffloadTest-vk :: Feature/ConstantBufferT/nested.test OffloadTest-vk :: Feature/MaximalReconvergence/loop_peeling.test OffloadTest-vk :: Feature/Semantics/CullDistance.test OffloadTest-vk :: Feature/TypedBuffer/64bit-scalar.test OffloadTest-vk :: Graphics/IsolineDomainTessellation.test OffloadTest-vk :: Graphics/gs_point_to_quad.test OffloadTest-vk :: Graphics/gs_selective_output.test OffloadTest-vk :: Tools/Offloader/BufferFormats.test OffloadTest-vk :: WaveOps/QuadReadAcrossDiagonal.int64.test OffloadTest-vk :: WaveOps/QuadReadAcrossX.int64.test OffloadTest-vk :: WaveOps/QuadReadAcrossY.int64.test OffloadTest-vk :: WaveOps/WaveActiveAllEqual.int64.test OffloadTest-vk :: WaveOps/WaveActiveBitAnd.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitAnd.int64.test OffloadTest-vk :: WaveOps/WaveActiveBitOr.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitOr.int64.test OffloadTest-vk :: WaveOps/WaveActiveBitXor.convergence.test OffloadTest-vk :: WaveOps/WaveActiveBitXor.int64.test OffloadTest-vk :: WaveOps/WavePrefixProduct.int64.test OffloadTest-vk :: WaveOps/WavePrefixSum.32.test OffloadTest-vk :: WaveOps/WavePrefixSum.fp16.test OffloadTest-vk :: WaveOps/WavePrefixSum.int64.test ******************** Unexpectedly Passed Tests (3): OffloadTest-vk :: Feature/HLSLLib/cos.nan-inf-denorm.32.test OffloadTest-vk :: Feature/HLSLLib/sin.nan-inf-denorm.32.test OffloadTest-vk :: WaveOps/WaveActiveSum.int16.test

farzonl · 2026-08-28T16:54:47Z


-# UNSUPPORTED: Clang && Vulkan
-# Clang-Vulkan is unsupported because of two validation errors
-# This issue tracks its resolution: https://github.com/llvm/offload-test-suite/issues/285


This issue is closed as not planned if support was added is that incorrect?

I changed it to closed as completed - the issue really didn't have enough information to determine what fixed it and I usually don't close things as completed unless I can verify that. It does seem that whatever was wrong is fixed now though, so sure.

farzonl · 2026-08-28T16:56:33Z

 # 16/64 bit firstbithigh doesn't have a DXC-Vulkan lowering
 # Unsupported https://github.com/microsoft/DirectXShaderCompiler/blob/main/tools/clang/test/CodeGenSPIRV/intrinsics.firstbitlow.64bit.hlsl
-# UNSUPPORTED: DXC && Vulkan
+# XFAIL: DXC && Vulkan


Should you add the DXC issue tracking this work here in a comment?

I pointed all of these at microsoft/DirectXShaderCompiler#8536, which technically only mentions 1 of the 4 cases but it seems like a reasonable enough place to track all 4.

farzonl · 2026-08-28T16:58:31Z


 # DXC + Vulkan does not support multi-dimensional resource arrays
-# UNSUPPORTED: DXC && Vulkan
+# XFAIL: DXC && Vulkan


I think if we are going to call these xfails now instead of unsupported we need an issue linked. If DXC was never going to get this feature, then unsupported seems to make more sense.

I do think if somebody implemented this we'd accept the patch, but unless someone comes along actually needing it I don't think it's worth filing an issue about. I've put it back to UNSUPPORTED.

farzonl · 2026-08-28T17:00:14Z


 # DXC + Vulkan does not support global structures containing buffers.
-# UNSUPPORTED: DXC && Vulkan
+# XFAIL: DXC && Vulkan


Same comment I think if we are going to call these xfails now instead of unsupported we need an issue linked. If DXC was never going to get this feature, then unsupported seems to make more sense.

Same thing here - moved it back to UNSUPPORTED.

bogner · 2026-08-28T18:55:45Z

There are two new test failures that I need to track down or file issues for:

Clang QC Vulkan countbits.64 somehow gets 6 instead of 3 when counting the bits of 0x2A.
Clang Intel D3D WaveReadLaneAt.mtx is crashing the driver. Could be [Intel Xe][WaveOps] Many of the waveOps intrinsics are failing on the Intel B50 GPU #1293 but it's hard to confirm

These all should be XFAILs, but they were marked as unsupported. Many of these pass now, so in those cases I've just removed them. There are two more interesting cases: - `countbits.16` had a mistake in the test and is failing for clang. I filed llvm/llvm-project#219327 to track the clang failure. - `SwitchBranchAttr` and `SwitchFlattenAttr` had the attribute in the wrong place and weren't actually testing what they said they were.

These don't have issues to associate with the XFAILs and we're extremely unlikely to prioritize the work, so UNSUPPORTED is probably more appropriate.

bogner

These tests pass on KosmicKrisp and MoltenVK

Co-authored-by: Justin Bogner <mail@justinbogner.com>

bogner · 2026-09-03T18:02:17Z

Test failures are all unrelated matrix max/min tests that have been resolved upstream

These all should be XFAILs, but they were marked as unsupported. Many of these pass now, so in those cases I've just removed them. There are two more interesting cases: - `countbits.16` had a mistake in the test and is failing for clang. I filed llvm/llvm-project#219327 to track the clang failure. - `SwitchBranchAttr` and `SwitchFlattenAttr` had the attribute in the wrong place and weren't actually testing what they said they were.

bogner added the test-all When applied to a PR this will opt-in to additional pre-merge test configurations.. label Aug 28, 2026

bob80905 reviewed Aug 28, 2026

View reviewed changes

Comment thread test/Feature/HLSLLib/countbits.16.test

Comment thread test/WaveOps/WaveActiveBitOr.int64.test

farzonl reviewed Aug 28, 2026

View reviewed changes

Comment thread test/Feature/HLSLLib/firstbithigh.64.test

farzonl reviewed Aug 28, 2026

View reviewed changes

Comment thread test/Feature/HLSLLib/firstbitlow.16.test

farzonl reviewed Aug 28, 2026

View reviewed changes

Comment thread test/Feature/HLSLLib/firstbitlow.64.test

farzonl reviewed Aug 28, 2026

View reviewed changes

bogner added 5 commits September 2, 2026 12:34

fixup: XFAIL also applies to metal

28d5221

fixup: link to firstbit dxc issue

63bbea9

fixup: Move two tests back to unsupported

b6ecdc2

These don't have issues to associate with the XFAILs and we're extremely unlikely to prioritize the work, so UNSUPPORTED is probably more appropriate.

fixup: Add WaveReadLaneAt.mtx xfail

988e45f

bogner force-pushed the 2026-08-27-support-unsupported branch from d0fee48 to 988e45f Compare September 2, 2026 19:34

farzonl reviewed Sep 3, 2026

View reviewed changes

Comment thread test/Basic/Matrix/matrix_bool_and_operator_thread_group.test Outdated

farzonl reviewed Sep 3, 2026

View reviewed changes

Comment thread test/Basic/Matrix/matrix_bool_or_operator.test Outdated

farzonl reviewed Sep 3, 2026

View reviewed changes

Comment thread test/Basic/Matrix/matrix_bool_and_operator.test Outdated

bogner commented Sep 3, 2026

View reviewed changes

Comment thread test/Basic/Matrix/matrix_bool_and_operator.test Outdated

Comment thread test/Basic/Matrix/matrix_bool_and_operator_thread_group.test Outdated

Comment thread test/Basic/Matrix/matrix_bool_or_operator.test Outdated

Remove UNSUPPORTED for KosmicKrisp/MoltenVK

cbddf38

Co-authored-by: Justin Bogner <mail@justinbogner.com>

farzonl approved these changes Sep 3, 2026

View reviewed changes

bogner merged commit 5e8c924 into llvm:main Sep 3, 2026
38 of 57 checks passed

Uh oh!

Conversation

bogner commented Aug 28, 2026

Uh oh!

Uh oh!

Uh oh!

farzonl Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Full Results:

KosmicKrisp Clang

KosmicKrisp DXC

MoltenVK Clang

MoltenVK DXC

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farzonl Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

farzonl Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bogner Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bogner commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bogner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bogner commented Sep 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

farzonl Aug 28, 2026 •

edited

Loading

farzonl Aug 28, 2026 •

edited

Loading

farzonl Aug 28, 2026 •

edited

Loading

bogner Aug 28, 2026 •

edited

Loading

bogner commented Aug 28, 2026 •

edited

Loading