Skip to content

Commit

Permalink
Add Metal data type table
Browse files Browse the repository at this point in the history
  • Loading branch information
YuAo committed Apr 11, 2022
1 parent bbe31e5 commit eecb717
Show file tree
Hide file tree
Showing 10 changed files with 426 additions and 8 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.docc-build/
.swiftpm/
.DS_Store
/.build
.build
/Packages
/*.xcodeproj
xcuserdata/
84 changes: 83 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,71 @@ Content of the `SARC` tag:
| 0...n | NULL-terminated C-style string | ID of the source code archive |
| n... | BZh | Bzip2 compressed source code archive |

### Metal Data Type Table

| Value | Type | Value | Type |
| ----- | ---- | ----- | ---- |
| 0x00 | None | 0x01 | Struct |
| 0x02 | Array | 0x03 | Float |
| 0x04 | Float2 | 0x05 | Float3 |
| 0x06 | Float4 | 0x07 | Float2x2 |
| 0x08 | Float2x3 | 0x09 | Float2x4 |
| 0x0A | Float3x2 | 0x0B | Float3x3 |
| 0x0C | Float3x4 | 0x0D | Float4x2 |
| 0x0E | Float4x3 | 0x0F | Float4x4 |
| 0x10 | Half | 0x11 | Half2 |
| 0x12 | Half3 | 0x13 | Half4 |
| 0x14 | Half2x2 | 0x15 | Half2x3 |
| 0x16 | Half2x4 | 0x17 | Half3x2 |
| 0x18 | Half3x3 | 0x19 | Half3x4 |
| 0x1A | Half4x2 | 0x1B | Half4x3 |
| 0x1C | Half4x4 | 0x1D | Int |
| 0x1E | Int2 | 0x1F | Int3 |
| 0x20 | Int4 | 0x21 | UInt |
| 0x22 | UInt2 | 0x23 | UInt3 |
| 0x24 | UInt4 | 0x25 | Short |
| 0x26 | Short2 | 0x27 | Short3 |
| 0x28 | Short4 | 0x29 | UShort |
| 0x2A | UShort2 | 0x2B | UShort3 |
| 0x2C | UShort4 | 0x2D | Char |
| 0x2E | Char2 | 0x2F | Char3 |
| 0x30 | Char4 | 0x31 | UChar |
| 0x32 | UChar2 | 0x33 | UChar3 |
| 0x34 | UChar4 | 0x35 | Bool |
| 0x36 | Bool2 | 0x37 | Bool3 |
| 0x38 | Bool4 | 0x3A | Texture |
| 0x3B | Sampler | 0x3C | Pointer |
| 0x3E | R8Unorm | 0x3F | R8Snorm |
| 0x40 | R16Unorm | 0x41 | R16Snorm |
| 0x42 | RG8Unorm | 0x43 | RG8Snorm |
| 0x44 | RG16Unorm | 0x45 | RG16Snorm |
| 0x46 | RGBA8Unorm | 0x47 | RGBA8Unorm_sRGB |
| 0x48 | RGBA8Snorm | 0x49 | RGBA16Unorm |
| 0x4A | RGBA16Snorm | 0x4B | RGB10A2Unorm |
| 0x4C | RG11B10Float | 0x4D | RGB9E5Float |
| 0x4E | RenderPipeline | 0x4F | ComputePipeline |
| 0x50 | IndirectCommandBuffer | 0x51 | Long |
| 0x52 | Long2 | 0x53 | Long3 |
| 0x54 | Long4 | 0x55 | ULong |
| 0x56 | ULong2 | 0x57 | ULong3 |
| 0x58 | ULong4 | 0x59 | Double |
| 0x5A | Double2 | 0x5B | Double3 |
| 0x5C | Double4 | 0x5D | Float8 |
| 0x5E | Float16 | 0x5F | Half8 |
| 0x60 | Half16 | 0x61 | Int8 |
| 0x62 | Int16 | 0x63 | UInt8 |
| 0x64 | UInt16 | 0x65 | Short8 |
| 0x66 | Short16 | 0x67 | UShort8 |
| 0x68 | UShort16 | 0x69 | Char8 |
| 0x6A | Char16 | 0x6B | UChar8 |
| 0x6C | UChar16 | 0x6D | Long8 |
| 0x6E | Long16 | 0x6F | ULong8 |
| 0x70 | ULong16 | 0x71 | Double8 |
| 0x72 | Double16 | 0x73 | VisibleFunctionTable |
| 0x74 | IntersectionFunctionTable | 0x75 | PrimitiveAccelerationStructure |
| 0x76 | InstanceAccelerationStructure | 0x77 | Bool8 |
| 0x78 | Bool16 | | |

## ❤️ Contributing

If you think there's a mistake, please open an issue. You can also choose to open a pull request with the failure test included.
Expand Down Expand Up @@ -385,11 +450,28 @@ I also found a few things interesting in this process:
### Updates
**Apr 10, 2022**
Tags like `LAYR`, `VATY`, `CNST`, etc., contain `UInt8` values of Metal data types. The corresponding description for each data type value can be retrieved using a private class in Metal.framework - [MTLTypeInternal](https://github.com/nst/iOS-Runtime-Headers/blob/fbb634c78269b0169efdead80955ba64eaaa2f21/Frameworks/Metal.framework/MTLTypeInternal.h)
```objective-c
id value = [[NSClassFromString(@"MTLTypeInternal") alloc] initWithDataType:0x06];
NSLog(@"%@", value.description); // MTLDataTypeFloat4
```
I created a command line tool to generate the Metal data type table.
```shell
cd Utilities
swift run metal-data-type-tools gen-markdown --columns 2 # generate a markdown table
swift run metal-data-type-tools gen-swift # generate a Swift enum for Metal data types.
```
**Mar 31, 2022**
The `air-lld` (`Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/metal/ios/bin/air-lld`) also provides a lot of information about how the `metallib` file is built. Some section names and descriptions are updated.
```
```cpp
int __ZN4llvm3air20MetalLibObjectWriter5writeEv() {
r14 = rdi;
rax = llvm::air::MetalLibObjectWriter::writeHeader();
Expand Down
127 changes: 127 additions & 0 deletions Sources/MetalLibraryArchive/MetalDataType.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
//
// MetalDataType.swift
// Generated on 2022-04-10 15:38:24 +0000
//

public enum MetalDataType: UInt8, Hashable, CaseIterable {
case none = 0x00
case `struct` = 0x01
case array = 0x02
case float = 0x03
case float2 = 0x04
case float3 = 0x05
case float4 = 0x06
case float2x2 = 0x07
case float2x3 = 0x08
case float2x4 = 0x09
case float3x2 = 0x0A
case float3x3 = 0x0B
case float3x4 = 0x0C
case float4x2 = 0x0D
case float4x3 = 0x0E
case float4x4 = 0x0F
case half = 0x10
case half2 = 0x11
case half3 = 0x12
case half4 = 0x13
case half2x2 = 0x14
case half2x3 = 0x15
case half2x4 = 0x16
case half3x2 = 0x17
case half3x3 = 0x18
case half3x4 = 0x19
case half4x2 = 0x1A
case half4x3 = 0x1B
case half4x4 = 0x1C
case int = 0x1D
case int2 = 0x1E
case int3 = 0x1F
case int4 = 0x20
case uint = 0x21
case uint2 = 0x22
case uint3 = 0x23
case uint4 = 0x24
case short = 0x25
case short2 = 0x26
case short3 = 0x27
case short4 = 0x28
case ushort = 0x29
case ushort2 = 0x2A
case ushort3 = 0x2B
case ushort4 = 0x2C
case char = 0x2D
case char2 = 0x2E
case char3 = 0x2F
case char4 = 0x30
case uchar = 0x31
case uchar2 = 0x32
case uchar3 = 0x33
case uchar4 = 0x34
case bool = 0x35
case bool2 = 0x36
case bool3 = 0x37
case bool4 = 0x38
case texture = 0x3A
case sampler = 0x3B
case pointer = 0x3C
case r8Unorm = 0x3E
case r8Snorm = 0x3F
case r16Unorm = 0x40
case r16Snorm = 0x41
case rg8Unorm = 0x42
case rg8Snorm = 0x43
case rg16Unorm = 0x44
case rg16Snorm = 0x45
case rgba8Unorm = 0x46
case rgba8Unorm_sRGB = 0x47
case rgba8Snorm = 0x48
case rgba16Unorm = 0x49
case rgba16Snorm = 0x4A
case rgb10A2Unorm = 0x4B
case rg11B10Float = 0x4C
case rgb9E5Float = 0x4D
case renderPipeline = 0x4E
case computePipeline = 0x4F
case indirectCommandBuffer = 0x50
case long = 0x51
case long2 = 0x52
case long3 = 0x53
case long4 = 0x54
case ulong = 0x55
case ulong2 = 0x56
case ulong3 = 0x57
case ulong4 = 0x58
case double = 0x59
case double2 = 0x5A
case double3 = 0x5B
case double4 = 0x5C
case float8 = 0x5D
case float16 = 0x5E
case half8 = 0x5F
case half16 = 0x60
case int8 = 0x61
case int16 = 0x62
case uint8 = 0x63
case uint16 = 0x64
case short8 = 0x65
case short16 = 0x66
case ushort8 = 0x67
case ushort16 = 0x68
case char8 = 0x69
case char16 = 0x6A
case uchar8 = 0x6B
case uchar16 = 0x6C
case long8 = 0x6D
case long16 = 0x6E
case ulong8 = 0x6F
case ulong16 = 0x70
case double8 = 0x71
case double16 = 0x72
case visibleFunctionTable = 0x73
case intersectionFunctionTable = 0x74
case primitiveAccelerationStructure = 0x75
case instanceAccelerationStructure = 0x76
case bool8 = 0x77
case bool16 = 0x78
}

17 changes: 11 additions & 6 deletions Tests/MetalLibraryArchiveTests/Tests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,7 @@ class MetalLibraryArchiveTests_macOSSDK: XCTestCase {
using namespace metal;
struct ControlPoint {
float3 position [[attribute(0)]];
float4 position [[attribute(0)]];
float3 normal [[attribute(1)]];
};
Expand All @@ -473,10 +473,15 @@ class MetalLibraryArchiveTests_macOSSDK: XCTestCase {
XCTAssertEqual(archive.functions.count, 1)
XCTAssertEqual(archive.targetPlatform, sdk.targetPlatform)
XCTAssertEqual(archive.libraryType, .executable)
let tessellationTag = try XCTUnwrap(archive.functions[0].tags.first(where: { $0.name == "TESS" }))
let tessellationTag = try XCTUnwrap(archive.functions.first?.tags.first(where: { $0.name == "TESS" }))
XCTAssertEqual(tessellationTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[0] }), 4 << 2 | 2)
XCTAssert(archive.functions[0].publicMetadataTags.contains(where: { $0.name == "VATT" }))
XCTAssert(archive.functions[0].publicMetadataTags.contains(where: { $0.name == "VATY" }))

let vatyTag = try XCTUnwrap(archive.functions.first?.publicMetadataTags.first(where: { $0.name == "VATY" }))
XCTAssertEqual(vatyTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt16.self)[0] }), 2) // count
XCTAssertEqual(vatyTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[2] }), MetalDataType.float4.rawValue) //position
XCTAssertEqual(vatyTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[3] }), MetalDataType.float3.rawValue) //normal
}

func testFuntionConstants() throws {
Expand Down Expand Up @@ -534,8 +539,8 @@ class MetalLibraryArchiveTests_macOSSDK: XCTestCase {
XCTAssertEqual(archive.functions.count, 1)
XCTAssertEqual(archive.targetPlatform, sdk.targetPlatform)
XCTAssertEqual(archive.libraryType, .executable)
let layerTag = try XCTUnwrap(archive.functions[0].tags.first(where: { $0.name == "LAYR" }))
XCTAssertEqual(layerTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[0] }), 0x21)
let layerTag = try XCTUnwrap(archive.functions.first?.tags.first(where: { $0.name == "LAYR" }))
XCTAssertEqual(layerTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[0] }), MetalDataType.uint.rawValue)
}

func testLayeredRendering_ushort() throws {
Expand All @@ -557,8 +562,8 @@ class MetalLibraryArchiveTests_macOSSDK: XCTestCase {
XCTAssertEqual(archive.functions.count, 1)
XCTAssertEqual(archive.targetPlatform, sdk.targetPlatform)
XCTAssertEqual(archive.libraryType, .executable)
let layerTag = try XCTUnwrap(archive.functions[0].tags.first(where: { $0.name == "LAYR" }))
XCTAssertEqual(layerTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[0] }), 0x29)
let layerTag = try XCTUnwrap(archive.functions.first?.tags.first(where: { $0.name == "LAYR" }))
XCTAssertEqual(layerTag.content.withUnsafeBytes({ $0.bindMemory(to: UInt8.self)[0] }), MetalDataType.ushort.rawValue)
}

func testSourceArchives_executable() throws {
Expand Down
16 changes: 16 additions & 0 deletions Utilities/Package.resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"object": {
"pins": [
{
"package": "swift-argument-parser",
"repositoryURL": "https://github.com/apple/swift-argument-parser",
"state": {
"branch": null,
"revision": "82905286cc3f0fa8adc4674bf49437cab65a8373",
"version": "1.1.1"
}
}
]
},
"version": 1
}
24 changes: 24 additions & 0 deletions Utilities/Package.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// swift-tools-version:5.4
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
name: "Utilities",
platforms: [.macOS(.v10_13)],
products: [
.executable(name: "metal-data-type-tools", targets: ["MetalDataTypeTools"])
],
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.0.0")
],
targets: [
.target(name: "MetalDataTypeInternal"),
.executableTarget(
name: "MetalDataTypeTools",
dependencies: [
"MetalDataTypeInternal",
.product(name: "ArgumentParser", package: "swift-argument-parser"),
])
]
)
13 changes: 13 additions & 0 deletions Utilities/Sources/MetalDataTypeInternal/MetalDataTypeInternal.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
//
// a.m
//
//
// Created by YuAo on 2022/4/10.
//

#import "MetalDataTypeInternal.h"

id<MetalDataTypeObject> MetalDataTypeObjectCreate(unsigned long long type) {
id value = [[NSClassFromString(@"MTLTypeInternal") alloc] initWithDataType:type];
return value;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
//
// Header.h
//
//
// Created by YuAo on 2022/4/10.
//

@import Foundation;
@import Metal;

NS_ASSUME_NONNULL_BEGIN

// MTLTypeInternal
@protocol MetalDataTypeObject

- (id)initWithDataType:(unsigned long long)arg1;

@property (nonatomic, readonly) unsigned long long dataType;

@property (nonatomic, readonly) NSString * description;

@end

FOUNDATION_EXPORT id<MetalDataTypeObject> MetalDataTypeObjectCreate(unsigned long long type);

NS_ASSUME_NONNULL_END
Loading

0 comments on commit eecb717

Please sign in to comment.