Add compute_shaders_with_tensors sample

[matthewsloyanARM]

Description

This sample demonstrates the usage of the VK_ARM_tensors and VK_ARM_data_graph extensions for machine learning inference. These allow you to define a neural network with TOSA operations and then execute inferences using hardware acceleration.

This is the third in a series of samples which shows a user how they can use compute shaders to produce inputs to a simple neural network. There is also a post-processing compute shader that modifies the output tensor.

The first sample can be found here: #1394

Build & run guide

If you would like to try out this new sample locally, we provide an emulation layer which allows them to run. This can be found here: https://github.com/arm/ai-ml-emulation-layer-for-vulkan. There is also a pre-built Windows and Linux package available here: https://github.com/arm/ai-ml-emulation-layer-for-vulkan/releases/tag/v0.8.0 or you can grab it through our tutorial: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/. This tutorial is for this first sample already merged (simple_tensor_and_data_graph), so it shows you everything needed to try it out. The guide is only for Windows though, but the README in the ai-ml-emulation-layer-for-vulkan repository provides a build and usage guide for Linux.

General Checklist:

Please ensure the following points are checked:

• My code follows the coding style
• I have reviewed file licenses
• I have commented any added functions (in line with Doxygen)
• I have commented any code that could be hard to understand
• My changes do not add any new compiler warnings
• My changes do not add any new validation layer errors or warnings
• I have used existing framework/helper functions where possible
• My changes do not add any regressions
• I have tested every sample to ensure everything runs correctly
• This PR describes the scope and expected impact of the changes I am making

Note: The Samples CI runs a number of checks including:

• I have updated the header Copyright to reflect the current year (CI build will fail if Copyright is out of date)
• My changes build on Windows, Linux, macOS and Android. Otherwise I have documented any exceptions

If this PR contains framework changes:

• I did a full batch run using the batch command line argument to make sure all samples still work properly

Sample Checklist

If your PR contains a new or modified sample, these further checks must be carried out in addition to the General Checklist:

• I have tested the sample on at least one compliant Vulkan implementation
• If the sample is vendor-specific, I have tagged it appropriately
• I have stated on what implementation the sample has been tested so that others can test on different implementations and platforms
• Any dependent assets have been merged and published in downstream modules
• For new samples, I have added a paragraph with a summary to the appropriate chapter in the readme of the folder that the sample belongs to e.g. api samples readme
• For new samples, I have added a tutorial README.md file to guide users through what they need to know to implement code using this feature. For example, see conditional_rendering
• For new samples, I have added a link to the Antora navigation so that the sample will be listed at the Vulkan documentation site

[asuessenbach]

This sample looks great! I especially like the good comments on all relevant aspects.

Just a couple of comments...

[SaschaWillems]

Sample works fine for me, but has several validation errors:

[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://docs.vulkan.org/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://docs.vulkan.org/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://docs.vulkan.org/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[error] 115483881 - VUID-VkShaderModuleCreateInfo-pCode-08740: vkCreateShaderModule(): SPIR-V Capability GraphARM was declared, but one of the following requirements is required (VkPhysicalDeviceDataGraphFeaturesARM::dataGraph).
The Vulkan spec states: If pCode is a pointer to SPIR-V code, and pCode declares any of the capabilities listed in the SPIR-V Environment appendix, one of the corresponding requirements must be satisfied (https://docs.vulkan.org/spec/latest/chapters/shaders.html#VUID-VkShaderModuleCreateInfo-pCode-08740)
[error] 167135271 - VUID-vkCreateDataGraphPipelinesARM-dataGraph-09760: vkCreateDataGraphPipelinesARM(): dataGraph feature is not enabled.
The Vulkan spec states: The dataGraph feature must be enabled (https://docs.vulkan.org/spec/latest/chapters/VK_ARM_data_graph/graphs.html#VUID-vkCreateDataGraphPipelinesARM-dataGraph-09760)
[error] 1553436542 - VUID-vkGetDataGraphPipelineSessionMemoryRequirementsARM-bindPoint-09784: vkGetDataGraphPipelineSessionMemoryRequirementsARM(): pInfo->bindPoint (VK_DATA_GRAPH_PIPELINE_SESSION_BIND_POINT_TRANSIENT_ARM) not found in session memory requirements.
The Vulkan spec states: The bindPoint member of pInfo must have been returned as part of a VkDataGraphPipelineSessionBindPointRequirementARM whose bindPointType member is VK_DATA_GRAPH_PIPELINE_SESSION_BIND_POINT_TYPE_MEMORY_ARM by a prior call to vkGetDataGraphPipelineSessionMemoryRequirementsARM for the session member of pInfo (https://docs.vulkan.org/spec/latest/chapters/VK_ARM_data_graph/graphs.html#VUID-vkGetDataGraphPipelineSessionMemoryRequirementsARM-bindPoint-09784)

Using the latest SDK.

[matthewsloyanARM]

@asuessenbach I have addressed your comments, thank you for review and the suggestions!

@SaschaWillems Thank you for trying this out! I will look into this with the latest SDK on our side. The tensor and data graph still isn't fully supported in the public validation layer yet, so this might be the cause.

I have also added the guide to downloading and using the ML Extensions for Vulkan Emulation Layer to the description so other can try it out.

[gpx1000]

I'm getting a few more VVL issues:

[debug] Building #0 cache object (N3vkb4core22HPPDescriptorSetLayoutE)
[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[error] 222824743 - VUID-vkCreateTensorARM-tensors-09832: vkCreateTensorARM(): tensors feature is not enabled.
The Vulkan spec states: The tensors feature must be enabled (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/resources.html#VUID-vkCreateTensorARM-tensors-09832)
[debug] Building #2 cache object (N3vkb12ShaderModuleE)
[debug] Building #1 cache object (N3vkb19DescriptorSetLayoutE)
[error] 115483881 - VUID-VkShaderModuleCreateInfo-pCode-08740: vkCreateShaderModule(): SPIR-V Capability GraphARM was declared, but one of the following requirements is required (VkPhysicalDeviceDataGraphFeaturesARM::dataGraph).
The Vulkan spec states: If pCode is a pointer to SPIR-V code, and pCode declares any of the capabilities listed in the SPIR-V Environment appendix, one of the corresponding requirements must be satisfied (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/shaders.html#VUID-VkShaderModuleCreateInfo-pCode-08740)
[error] 167135271 - VUID-vkCreateDataGraphPipelinesARM-dataGraph-09760: vkCreateDataGraphPipelinesARM(): dataGraph feature is not enabled.
The Vulkan spec states: The dataGraph feature must be enabled (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/VK_ARM_data_graph/graphs.html#VUID-vkCreateDataGraphPipelinesARM-dataGraph-09760)
[error] -1656326806 - VUID-VkDataGraphPipelineCreateInfoARM-dataGraphShaderModule-09886: vkCreateDataGraphPipelinesARM(): pCreateInfos[0].pNext<VkDataGraphPipelineShaderModuleCreateInfoARM> is in the pNext chain, but the dataGraphShaderModule feature is not enabled.
pNext chain: VkDataGraphPipelineCreateInfoARM::pNext -> [VkDataGraphPipelineShaderModuleCreateInfoARM].
The Vulkan spec states: If the dataGraphShaderModule feature is not enabled, a VkDataGraphPipelineShaderModuleCreateInfoARM structure must not be included in the pNext chain (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/VK_ARM_data_graph/graphs.html#VUID-VkDataGraphPipelineCreateInfoARM-dataGraphShaderModule-09886)
[error] 1553436542 - VUID-vkGetDataGraphPipelineSessionMemoryRequirementsARM-bindPoint-09784: vkGetDataGraphPipelineSessionMemoryRequirementsARM(): pInfo->bindPoint (VK_DATA_GRAPH_PIPELINE_SESSION_BIND_POINT_TRANSIENT_ARM) not found in session memory requirements.
The Vulkan spec states: The bindPoint member of pInfo must have been returned as part of a VkDataGraphPipelineSessionBindPointRequirementARM whose bindPointType member is VK_DATA_GRAPH_PIPELINE_SESSION_BIND_POINT_TYPE_MEMORY_ARM by a prior call to vkGetDataGraphPipelineSessionMemoryRequirementsARM for the session member of pInfo (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/VK_ARM_data_graph/graphs.html#VUID-vkGetDataGraphPipelineSessionMemoryRequirementsARM-bindPoint-09784)

[error] -181972258 - VUID-vkCmdDispatch-pDescription-09900: vkCmdDispatch(): the tensor descriptor [VkDescriptorSet 0x2d000000002d, Set 0, Binding 0, Index 0, variable "output_tensor"] is using tensor VkTensorARM 0x1e000000001e created with usage VK_TENSOR_USAGE_DATA_GRAPH_BIT_ARM, which doesn't include VK_TENSOR_USAGE_SHADER_BIT_ARM.
The Vulkan spec states: If a VK_DESCRIPTOR_TYPE_TENSOR_ARM descriptor is accessed as a result of this command, then the underlying VkTensorARM object must have been created with a VkTensorCreateInfoARM::pDescription whose usage member contained VK_TENSOR_USAGE_SHADER_BIT_ARM (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/dispatch.html#VUID-vkCmdDispatch-pDescription-09900)
[error] -181972258 - VUID-vkCmdDispatch-pDescription-09900: vkCmdDispatch(): the tensor descriptor [VkDescriptorSet 0x410000000041, Set 0, Binding 0, Index 0, variable "input_tensor"] is using tensor VkTensorARM 0x1e000000001e created with usage VK_TENSOR_USAGE_DATA_GRAPH_BIT_ARM, which doesn't include VK_TENSOR_USAGE_SHADER_BIT_ARM.
The Vulkan spec states: If a VK_DESCRIPTOR_TYPE_TENSOR_ARM descriptor is accessed as a result of this command, then the underlying VkTensorARM object must have been created with a VkTensorCreateInfoARM::pDescription whose usage member contained VK_TENSOR_USAGE_SHADER_BIT_ARM (https://vulkan.lunarg.com/doc/view/1.4.335.0/linux/antora/spec/latest/chapters/dispatch.html#VUID-vkCmdDispatch-pDescription-09900)

Can I recommend adding some instructions for Linux? Users don't need to have to build from source to get the layer to work. This is what I did (this run is tested in Ubuntu):
Download the tagged release appropiate to your system from here: https://github.com/arm/ai-ml-emulation-layer-for-vulkan/releases/tag/v0.8.0
Now, you have a few options, you may add this directly to the VulkanSDK, or, better, you may add to the VulkanSDK, or safer solution, add it to your VK_LAYER environment variable. Run vkconfig same instructions as Windows from here.

It might even be a worthwhile idea to make a few scripts. Just thinking it's probably best to recommend end users use tagged releases rather than clone the repo.

Other than those recommendations; this works great in Windows, and Linux. I haven't tested Android yet, but I will.

[matthewsloyanARM]

Thank you @gpx1000 for the review, we really appreciate the thoughts. I have updated the README.md in the tensor_and_data_graph and the description of this pull request to mention that the emulation layer is now available pre-built on GitHub. I have also generalized the section on using the vkconfig as an alternative to exporting the variables. Thanks for pointing that out! Also thanks for trying on Windows and Linux. Just to note we don't currently support Android which is mentioned in the README, but we're working on it.

Regarding the validation layers, sorry about the delay. I have addressed majority of the issues. The updated extensions weren't being requested, so this fixes the tensor and shaderModule related errors. However, there are two remaining (see below) due to a bug in the emulation layer. This is currently being investigated from our side,

[error] 1211499479 - VUID-VkTensorDescriptionARM-pDimensions-09883:
The VkPhysicalDeviceTensorPropertiesARM values (maxPerDimensionTensorElements etc) aren't being initialized correctly in the emulation layer, so it produces these errors for all tensors. This should happen automatically. These values are only currently used in the validation layers so they don't affect the running of the sample.

[error] 1553436542 - VUID-vkGetDataGraphPipelineSessionMemoryRequirementsARM-bindPoint-09784:
This still occurs even when using the bindPoint returned from vkGetDataGraphPipelineSessionMemoryRequirementsARM. I have validated that the correct type is passed and there is a check in the code for this already, so I haven't modified the code for now. This will be addressed properly and fixed in a later version.

@SaschaWillems, @gpx1000 Are you both okay with this for now, as it doesn't affect the running sample? We have two more samples to push and some general improvements, so these two errors will be addressed once the issues are sorted in the emulation layer. I have also updated the checkbox to reflect this.

[SaschaWillems]

I'm totally fine with this. We can always fix outstanding issues after merging.