Skip to content

Commit d506312

Browse files
lucylqlucylq
authored
Documentation for the PTD file format
Differential Revision: D78019434 Pull Request resolved: #12316
1 parent edf25c4 commit d506312

File tree

3 files changed

+202
-3
lines changed

3 files changed

+202
-3
lines changed

‎docs/source/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ ExecuTorch provides support for:
7575
- [Platform Abstraction Layer](runtime-platform-abstraction-layer)
7676
#### Portable C++ Programming
7777
- [PTE File Format](pte-file-format)
78+
- [PTD File Format](ptd-file-format)
7879
#### API Reference
7980
- [Export to Executorch API Reference](export-to-executorch-api-reference)
8081
- [Executorch Runtime API Reference](executorch-runtime-api-reference)
@@ -196,6 +197,7 @@ runtime-backend-delegate-implementation-and-linking
196197
runtime-platform-abstraction-layer
197198
portable-cpp-programming
198199
pte-file-format
200+
ptd-file-format
199201
```
200202

201203
```{toctree}

‎docs/source/ptd-file-format.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# `.ptd` file format
2+
3+
ExecuTorch `.ptd` files are serialized as modified binary flatbuffer
4+
files with data segments appended. They provide a way to store named data using
5+
the FlatTensor format. Named data can be tensors or opaque blob data (usually for backends that do not expose data format).
6+
7+
Code related to the PTD file format is in the `//executorch/extension/flat_tensor/` directory.
8+
9+
```
10+
┌───────────────────────────────────┐
11+
│Standard flatbuffer header │
12+
├───────────────────────────────────┤
13+
│ExecuTorch extended header │
14+
├───────────────────────────────────┤
15+
│Flatbuffer-serialized metadata │
16+
│(FlatTensor) │
17+
│ │
18+
┌─ ├───────────────────────────────────┤
19+
│ │Padding │
20+
│ ├───────────────────────────────────┤
21+
│ │Data segment │
22+
│ │ │
23+
│ │ │
24+
│ ├───────────────────────────────────┤
25+
│ │Padding │
26+
Blobs ─┤ ├───────────────────────────────────┤
27+
│ │Data segment │
28+
│ │ │
29+
│ │ │
30+
│ ├───────────────────────────────────┤
31+
│ │Padding │
32+
│ ├───────────────────────────────────┤
33+
│ │... │
34+
└─ └───────────────────────────────────┘
35+
```
36+
37+
## Compatibility
38+
39+
PTD files are designed for storing named data that can be loaded by ExecuTorch
40+
models.
41+
42+
## Headers
43+
44+
PTD files can be recognized by the magic string at byte offset 4, beginning with `FT`
45+
and followed by two ASCII decimal digits (file identifier from the FlatBuffers schema).
46+
47+
PTD files have an extended header at byte offset 8, recognized by the magic string
48+
`FH01`. This header includes the size and offset information for both the
49+
flatbuffer-serialized metadata and the data segments that follow.
50+
51+
Note that this header is ExecuTorch-specific, but even when present it does not
52+
upset most flatbuffer-parsing code (apart from the rarely-used
53+
`GetBufferStartFromRootPointer()`).
54+
55+
All numbers are little-endian, regardless of the host system.
56+
57+
Header layout:
58+
```
59+
[0..3] uint32_t byte offset to the beginning of the flatbuffer root table.
60+
[4..7] File magic bytes: "FT" followed by two ASCII decimal digits. The digits
61+
correspond to the FlatBuffers file identifier.
62+
Extended header (always present):
63+
| [8..11] Extended header magic bytes: "FH01" - FlatTensor Header version 01.
64+
| [12..15] uint32_t size of this extended header in bytes, including the magic
65+
| header and this size field. Currently fixed at 40 bytes.
66+
| [16..23] uint64_t offset (from byte offset zero) to the start of the
67+
| flatbuffer data.
68+
| [24..31] uint64_t size of the flatbuffer-encoded tensor metadata in bytes.
69+
| [32..39] uint64_t offset (from byte offset zero) to the start of the first
70+
| data segment.
71+
| [40..47] uint64_t total size of all data segments in bytes.
72+
End of extended header.
73+
```
74+
75+
Example:
76+
```
77+
Offset to flatbuffer root (0x44)
78+
| File magic ("FT01")
79+
| | Extended header magic ("FH01")
80+
| | | Extended header size (0x28)
81+
vvvvvvvvvvv vvvvvvvvvvv vvvvvvvvvvv vvvvvvvvvvv
82+
0x0000 44 00 00 00 46 54 30 31 46 48 30 31 28 00 00 00
83+
0x0010 30 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00
84+
0x0020 30 01 00 00 00 00 00 00 20 00 00 00 00 00 00 00
85+
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
86+
| | Flatbuffer size (0x100)
87+
| | Segment data size (0x20)
88+
Segment base offset (0x130)
89+
```
90+
Note: this example comes from inspecting the ModuleAddMul.ptd file.
91+
```
92+
python -m test.models.export_program --modules "ModuleAddMul" --external-constants --outdir .
93+
94+
xxd -l 64 ModuleAddMulProgram.ptd
95+
```
96+
97+
## FlatTensor
98+
99+
See `//executorch/extension/flat_tensor/serialize/flat_tensor.fbs` for the
100+
FlatTensor flatbuffer schema.
101+
102+
The flatbuffer-encoded metadata follows the headers and contains:
103+
104+
- **Schema version**: Version information for compatibility.
105+
- **Data segments**: List of segment descriptors with offset and size information.
106+
- **Named data**: List of named data entries, each containing:
107+
- **Key**: String identifier for the data blob.
108+
- **Segment index**: Reference to the data segment containing the blob.
109+
- **Tensor layout**: Optional metadata including scalar type, sizes and dim order, if the data segment contains a tensor.
110+
111+
### Tensor Layout
112+
113+
If a data segment contains a canonical tensor, it may have associated layout information:
114+
- **Scalar type**: Data type (float32, int32, etc.) using ExecutorTorch scalar types.
115+
- **Sizes**: Dimensions of the tensor.
116+
- **Dim order**: Memory layout order specifying how dimensions are arranged in memory.
117+
118+
## Data segments
119+
120+
The `FlatTensor.segments` list in the metadata contains offset and size
121+
information about each data segment. Offsets in this list are relative to
122+
the segment base offset specified in the extended header.
123+
124+
Each segment contains:
125+
- **Offset**: Relative offset from the segment base offset.
126+
- **Size**: Size of the valid data in bytes (may be followed by padding).
127+
128+
## Named data access
129+
130+
Tensors are accessed by string keys through the `named_data` list. Each entry
131+
maps a string key to:
132+
1. A segment index pointing to the raw data.
133+
2. Optional tensor layout metadata, if the data segment contains a tensor.
134+
135+
This design allows:
136+
- Multiple named data blobs to reference the same data segment.
137+
- Access to tensor layout data without loading the entire blob.
138+
139+
## Usage
140+
141+
PTD files are used to store data outside of the PTE file. Some use-cases:
142+
- On-device training: checkpointing for model weights.
143+
- Deduplication: sharing model weights between multiple executable PTE files.
144+
- Flexible deployment: allow async updates between program and data.

‎extension/flat_tensor/README.md

Lines changed: 56 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,59 @@
11
## FlatTensor
22

3-
> [!IMPORTANT]
4-
> FlatTensor is still under development, and not ready to use.
3+
FlatTensor is a flatbuffer-based format for storing and loading data with string-based keys. The format provides efficient serialization and deserialization of data with metadata and supports C++ and Python APIs. FlatTensor files use the `.ptd` extension.
54

6-
FlatTensor is a flatbuffer-based format for storing and loading tensors. The format provides a way to store tensors keyed by string.
5+
Major usage is to store data outside of the PTE file for clean program-data separation. Stored data may be tensor data or opaque blob data (for backends that do not expose data format).
6+
7+
### Schema
8+
9+
[flat_tensor.fbs](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/flat_tensor.fbs) contains the [Flatbuffers](https://google.github.io/flatbuffers/) schema used to serialize ExecuTorch data files.
10+
11+
[flat_tensor_schema.py](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/flat_tensor_schema.py) contains the python definition of the schema types.
12+
13+
### C++ APIs
14+
15+
[serialize.h](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/serialize.h) contains the APIs to serialize a PTD file.
16+
17+
[flat_tensor_data_map.h](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/flat_tensor_data_map.h) contains the APIs to deserialize a PTD file and interact with it via the [named_data_map.h](https://github.com/pytorch/executorch/blob/main/runtime/core/named_data_map.h) interface.
18+
19+
### Python APIs
20+
21+
[serialize.py](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/serialize.py) contains the Python serialization and deserialization APIs.
22+
23+
### Alignment Considerations
24+
25+
**Segment alignment**: Data segments are aligned to this value. This is usually some multiple of 2. Specified in the [FlatTensorConfig](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/serialize.py#L96).
26+
27+
**Tensor alignment**: Tensors are aligned to this value. Specified in the [FlatTensorConfig](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/serialize/serialize.py#L96).
28+
29+
**Blob alignment**: Blobs (may not be canonical tensors) are aligned to this value. Alignment is specified when blobs are added to the [_named_data_store.py](https://github.com/pytorch/executorch/blob/main/exir/_serialize/_named_data_store.py#L48) and passed to serialize.py.
30+
31+
FlatTensor does not store alignment in the serialized file; the user must ensure the serialized and runtime-expected alignment correspond. The final alignment may be a larger multiple of the specified alignment, as multiple `NamedData` entries can point to a single `DataSegment`. For example:
32+
```
33+
BackendA: {key = key1, data = 0x100, alignment = 4}
34+
BackendB: {key = key2, data = 0x100, alignment = 8}
35+
```
36+
BackendA and BackendB are serializing the same bytes, so the data is deduplicated and the final alignment is the lcm of the two, in this case 8.
37+
38+
### Usage
39+
40+
**AoT**
41+
42+
To export a model as a PTE and PTD pair, see [export_program.py](https://github.com/pytorch/executorch/blob/main/test/models/export_program.py). Use the `--external-constants` argument to move all constants to the separate PTD file.
43+
```
44+
python -m test.models.export_program --modules "ModuleAddMul" --external-constants --outdir .
45+
```
46+
47+
To export a delegated model as PTE and PTD pair, see [export_delegated_program.py](https://github.com/pytorch/executorch/blob/main/test/models/export_delegated_program.py). Use the `--external-constants` argument to move all constants to the separate PTD file. Note, ModuleLinear is used here as linear is consumed by the XNNPACK backend.
48+
```
49+
python -m test.models.export_delegated_program --modules ModuleLinear --backend_id XnnpackBackend --external_constants --outdir .
50+
```
51+
52+
**Runtime**
53+
54+
The `ProgramDataSeparationTest` in [method_test.cpp](https://github.com/pytorch/executorch/blob/main/runtime/executor/test/method_test.cpp) demonstrates how to consume the PTD file at runtime.
55+
56+
For a backend example with XNNPACK, see [test_xnn_data_separation.cpp](https://github.com/pytorch/executorch/blob/main/backends/xnnpack/test/runtime/test_xnn_data_separation.cpp).
57+
58+
### Rules to ensure forward/backward compatibility
59+
See [executorch/schema/README.md](https://github.com/pytorch/executorch/blob/main/schema/README.md).

0 commit comments

Comments
 (0)