[LongVec] Add Elementwise execution tests by farzonl · Pull Request #1463 · llvm/offload-test-suite

farzonl · 2026-08-20T18:25:13Z

resolves #1462

Assisted with GPT-5.6 Sol via Copilot

bob80905

Is this the complete set of elementwise intrinsics, or are there more to come?
If there's going to be many more to come, I wonder if it would be better to find a way to consolidate and parameterize these tests, they are quite repetitive.

bob80905 · 2026-08-27T20:26:00Z

+# RUN: %dxc_target -T %if Clang %{ps_6_0%} %else %{ps_6_9%} -Fo %t-pixel.o %t/pixel.hlsl
+# RUN: %offloader %t/pipeline.yaml %t-vertex.o %t-pixel.o | FileCheck %s
+
+# CHECK: Name: Output


I don't think it is conventional to run filecheck in the offload test suites, and buffer comparisons are preferred. Is there a reason you use filecheck here?

Yes because the ddy\ddx tests are pixel shader specific so we are not writing to a buffer the way our compute shaders do. I could potentially try and use gold images instead and do image compares via imgdiff. Thats how many of the other ddy\ddx testing has been done, but that is a bit more painful to test locally.

I thought about this a bit more and my plan to replace these checks with imgdiff did not workout. Long vector usage on an image is hard to wrap my brain around. I think you are right it will just be easier to do buffer comparisons and avoid file check all together.

bob80905 · 2026-08-27T20:26:36Z

+# Unimplemented https://github.com/llvm/wg-hlsl/issues/467
+# XFAIL: Vulkan
+
+# CI machines don't have SM 6.9 support yet


Isn't this only true for metal at the moment?

These lines don't have to be removed, but can be removed now. essentially if you don't have a new enough agility sdk then this requies allows us to skip the tests. CI has been updated so only value is for folks that don't have latest sdk setup to not get a bunch of test fails. But thats probably not something we want to encourage for users testing their code via this project.

farzonl · 2026-09-02T17:26:29Z

So this is interesting some long vector tests just started passing on vulkan without any changes from us:

Unexpectedly Passed Tests (29):
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/acos.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/asin.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/atan.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/atan2.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/ceil.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/cos.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/cosh.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/degrees.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/exp.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/exp2.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/f16tof32.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/f32tof16.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/floor.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/lerp.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/log.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/log10.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/log2.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/max.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/min.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/pow.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/radians.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/rcp.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/round.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/sin.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/sinh.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/sqrt.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/tan.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/tanh.long-vector.32.test
  OffloadTest-clang-vk :: Feature/HLSLLib/long-vector/trunc.long-vector.32.test

bogner

Do we need to test multiple sizes of vector for every single intrinsic? I think a single call to each elementwise intrinsic should be sufficient to prove that they work. Having just one of these tests cover the separate 5/8/16 cases seems sufficient to me.

bogner · 2026-09-03T21:12:11Z

+  vector<float, 5> Input0_5;
+  for (uint I = 0; I < 5; ++I)
+    Input0_5[I] = In0[0 + I];
+  vector<float, 5> Result5 = acos(Input0_5);
+  for (uint I = 0; I < 5; ++I)
+    Out[0 + I] = Result5[I];


We could save a bit of typing here if we use ByteAddressBuffer rather than StructuredBuffer. Consider:

vector<float, 5> Input0_5 = In0.Load< vector<float, 5> >(0); vector<float, 5> Result5 = acos(Input0_5); Out.Store< vector<float, 5> >(0, Result5);

or even:

Out.Store< vector<float, 5> >(0, acos(In0.Load< vector<float, 5> >(0)));

I really like your suggestion, however while less code this generates newer DirectX instructions to handle loads and stores

If I do:

ByteAddressBuffer In0 : register(t0); RWByteAddressBuffer Out : register(u1); [numthreads(1,1,1)] void main() { Out.Store< vector<float, 16> >(0, asfloat(In0.Load< vector<uint, 16> >(0))); }

I generate https://hlsl.godbolt.org/z/hz846Tb4b

define void @main() { %Out_UAV_rawbuf = call %dx.types.Handle @dx.op.createHandleFromBinding(i32 217, %dx.types.ResBind { i32 1, i32 1, i32 0, i8 1 }, i32 1, i1 false), !dbg !32 ; line:6 col:45 ; CreateHandleFromBinding(bind,index,nonUniformIndex) %In0_texture_rawbuf = call %dx.types.Handle @dx.op.createHandleFromBinding(i32 217, %dx.types.ResBind zeroinitializer, i32 0, i1 false), !dbg !32 ; line:6 col:45 ; CreateHandleFromBinding(bind,index,nonUniformIndex) %1 = call %dx.types.Handle @dx.op.annotateHandle(i32 216, %dx.types.Handle %In0_texture_rawbuf, %dx.types.ResourceProperties { i32 11, i32 0 }), !dbg !32 ; line:6 col:45 ; AnnotateHandle(res,props) resource: ByteAddressBuffer %RawBufferVectorLoad = call %dx.types.ResRet.v16i32 @dx.op.rawBufferVectorLoad.v16i32(i32 303, %dx.types.Handle %1, i32 0, i32 undef, i32 4), !dbg !32 ; line:6 col:45 ; RawBufferVectorLoad(buf,index,elementOffset,alignment) %2 = extractvalue %dx.types.ResRet.v16i32 %RawBufferVectorLoad, 0, !dbg !32 ; line:6 col:45 %3 = bitcast <16 x i32> %2 to <16 x float>, !dbg !33 ; line:6 col:37 %4 = call %dx.types.Handle @dx.op.annotateHandle(i32 216, %dx.types.Handle %Out_UAV_rawbuf, %dx.types.ResourceProperties { i32 4107, i32 0 }), !dbg !34 ; line:6 col:3 ; AnnotateHandle(res,props) resource: RWByteAddressBuffer call void @dx.op.rawBufferVectorStore.v16f32(i32 304, %dx.types.Handle %4, i32 0, i32 undef, <16 x float> %3, i32 4), !dbg !34 ; line:6 col:3 ; RawBufferVectorStore(uav,index,elementOffset,value0,alignment) ret void, !dbg !35 ; line:7 col:1 }

The problematic instructions being @dx.op.rawBufferVectorLoad.v16i32 and @dx.op.rawBufferVectorStore.v16f32.

However if I do

StructuredBuffer<uint> In0 : register(t0); RWStructuredBuffer<float> Out : register(u1); [numthreads(1,1,1)] void main() { vector<uint, 16> Input0_16; for (uint I = 0; I < 16; ++I) Input0_16[I] = In0[13 + I]; vector<float, 16> Result16 = asfloat(Input0_16); for (uint I = 0; I < 16; ++I) Out[13 + I] = Result16[I]; }

we get a vector bitcast and a vector load store https://hlsl.godbolt.org/z/Ydonf11ra

The important part of these tests being the %3 = bitcast <16 x i32> %2 to <16 x float> I wonder if it is worth reducing the hlsl code if it forces us to have to xfail Metal for DXC?

We discussed offline. @bogner suggested just xfailing metal. I don't love this as metal is more convenient for me to test this feature on my local machine when using DXC and DXC does not support SPV_EXT_long_vector so we can't do local testing via SPIR-V either.

I do have the ability to remote into a windows machine, which I suppose is a fine way forward.

FWIW while I do think having some test coverage of the vector rawBufferVectorLoad/Store is important and that using ByteAddressBuffer simplifies the tests, I'd be okay with most of these doing the more awkward code if it makes your life better.

That said, the metal shaderconverter definitely doesn't support all of long vector yet - I'm fairly surprised that any of the operations that actually lower to a dx.op.* with long vectors work there at all.

farzonl · 2026-09-03T21:20:14Z

Do we need to test multiple sizes of vector for every single intrinsic? I think a single call to each elementwise intrinsic should be sufficient to prove that they work. Having just one of these tests cover the separate 5/8/16 cases seems sufficient to me.

Fair I wrote a script to generate these using DXC for test case results. That's why they all test the same 3 vector sizes:

https://gist.github.com/farzonl/63d53a32cc41f6b5af39b1a99ffd9a26

bogner

Generally looks pretty good. Mostly have a bunch of nitpicks and a question or two.

bogner · 2026-09-09T08:46:10Z

+# Note We fail for DXC because today Clang scalairzed and does
+# Not hit  IRErrorCodeUnsupportedInstruction on vector @dx.op.*


I find this sentence a bit confusing (in addition to the typo and the odd whitespace). How about:

Suggested change

# Note We fail for DXC because today Clang scalairzed and does

# Not hit IRErrorCodeUnsupportedInstruction on vector @dx.op.*

# Note that we only pass on clang because Clang scalarizes. We will hit

# IRErrorCodeUnsupportedInstruction once Clang implements SM 6.10.

just deleted it. Comment isn't needed captured it instead in the issue.

bogner · 2026-09-09T08:53:12Z

+  Out.Store<vector<double, 8> >(0, asdouble(In0.Load<vector<uint, 8> >(0),
+      In1.Load<vector<uint, 8> >(0)));


clang-format (once you convince it to use Style: c++03) says:

Suggested change

Out.Store<vector<double, 8> >(0, asdouble(In0.Load<vector<uint, 8> >(0),

In1.Load<vector<uint, 8> >(0)));

Out.Store<vector<double, 8> >(0, asdouble(In0.Load<vector<uint, 8> >(0),

In1.Load<vector<uint, 8> >(0)));

How did you do this when I run clang-format it changes up the yaml?

I have a clang-format-region keybinding set up in my editor, otherwise it's a bit of a pain. You could run clang-format on the split files in the build directory but you may run into issues with shaders that use semantics in the arguments to the entry point, as those aren't handled very well by clang-format yet.

I don't have that tried doing it with regular clang format. then tried writing a script that ran split file did the format then merged back, eventually I just gave up and updated my generator to just align multi argument intrinsics. I would rather not keep getting caught up on formatting for tests, but you let me know how it looks.

bogner · 2026-09-09T08:55:49Z

+    Data: [0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000]
+    #     [     2,    0.5,      0,      1,     -1,      2,    0.5,      0,      1,     -1,      2,    0.5,      0,      1,     -1,      2]


aside (as in we don't need to change it here): it's really annoying that we use hex for float16 - it'd be nice to just use literals for exactly representable values like we do for float32

bogner · 2026-09-09T09:00:10Z

+    Rule: BufferFloatEpsilon
+    Epsilon: 0.008


Why choose epsilon over ULPs here (and elsewhere), and why 0.008? Is this how the precision of these ops is specified?

bogner · 2026-09-09T09:01:19Z

+  Out.Store<vector<float, 5> >(0, clamp(In0.Load<vector<float, 5> >(0),
+      In1.Load<vector<float, 5> >(0),
+      In2.Load<vector<float, 5> >(0)));


Suggested change

Out.Store<vector<float, 5> >(0, clamp(In0.Load<vector<float, 5> >(0),

In1.Load<vector<float, 5> >(0),

In2.Load<vector<float, 5> >(0)));

Out.Store<vector<float, 5> >(0, clamp(In0.Load<vector<float, 5> >(0),

In1.Load<vector<float, 5> >(0),

In2.Load<vector<float, 5> >(0)));

bogner · 2026-09-09T09:15:35Z

I'm not really following what's happening in this test. I guess if clip(Input5) discarded the pixel then clip(Input8) and clip(Input16) are redundant, but also every value in each input is identical. Can you add some comments about what we're doing here and/or simplify the test?

bogner · 2026-09-09T09:20:36Z

+  Out.Store<vector<double, 8> >(0, fma(In0.Load<vector<double, 8> >(0),
+      In1.Load<vector<double, 8> >(0),
+      In2.Load<vector<double, 8> >(0)));


More formatting. I'm going to stop pointing these out now but there are quite a few.

resolves llvm#1462

…ment intrinsics formated

…ts on either intel or nvida gpu depending on intrinsic

bob80905 self-assigned this Aug 26, 2026

bob80905 reviewed Aug 27, 2026

View reviewed changes

farzonl force-pushed the long_vec_e-wise-tests branch from 20a1348 to e05c661 Compare September 2, 2026 15:18

farzonl force-pushed the long_vec_e-wise-tests branch 3 times, most recently from fab8ecb to 8d4aa85 Compare September 3, 2026 16:59

bogner reviewed Sep 3, 2026

View reviewed changes

farzonl mentioned this pull request Sep 8, 2026

[Metal] Does not support vectorized @dx.op.rawBufferVectorLoad and @dx.op.rawBufferVectorStore #1506

Open

farzonl force-pushed the long_vec_e-wise-tests branch from 8d4aa85 to 210c469 Compare September 8, 2026 19:53

bogner reviewed Sep 9, 2026

View reviewed changes

farzonl added 16 commits September 13, 2026 17:56

[LongVec] Add Elementwise execution tests

591c5cc

resolves llvm#1462

update long vector tests

8e0208a

update with new issue numbers

db5c398

enable some vulkan long vector tests

47462b6

Metal issue llvm#1495 also impacts firstbitlow

f262e24

enable more vulkan long vector tests

6c1c248

respond to Josh's feedback

961e4e5

switch to ByteAddressBuffer

4c291c3

use the pre 202x language mode syntax that does not do >>

8d28181

switch to ByteAddressBuffer needs to be disabled for metal and dxc

4b6b5c6

remove comment

cb28763

switch to BufferFloatULP

a6d3c80

enable the vulkan tests that are now passing

f72ada1

move graphics shaders to just test one type of long vector.

17841bf

modify generate_long_vector_intrinsic_tests.py to generate multi argu…

6907316

…ment intrinsics formated

switch fma to BufferFloatULP

e422c54

farzonl force-pushed the long_vec_e-wise-tests branch from 210c469 to e422c54 Compare September 14, 2026 16:24

add ZeroTolerance changes to the failing trig tests

872ec02

farzonl added 2 commits September 14, 2026 18:39

increase ULP tolerance range for trig functions because of large drif…

977dc23

…ts on either intel or nvida gpu depending on intrinsic

xfail fma, make trig functions have higher ulp for warp

ea993df

		# Note We fail for DXC because today Clang scalairzed and does
		# Not hit IRErrorCodeUnsupportedInstruction on vector @dx.op.*

		Out.Store<vector<double, 8> >(0, asdouble(In0.Load<vector<uint, 8> >(0),
		In1.Load<vector<uint, 8> >(0)));

		Data: [0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000, 0x3800, 0x0000, 0x3C00, 0xBC00, 0x4000]
		# [ 2, 0.5, 0, 1, -1, 2, 0.5, 0, 1, -1, 2, 0.5, 0, 1, -1, 2]

Uh oh!

Conversation

farzonl commented Aug 20, 2026

Uh oh!

bob80905 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farzonl Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farzonl commented Sep 2, 2026

Uh oh!

bogner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farzonl commented Sep 3, 2026

Uh oh!

bogner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

farzonl Sep 2, 2026 •

edited

Loading