-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathReaders.html
More file actions
291 lines (275 loc) · 17.8 KB
/
Copy pathReaders.html
File metadata and controls
291 lines (275 loc) · 17.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Developing lld Readers — lld 11 documentation</title>
<link rel="stylesheet" href="_static/llvm.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
<link rel="shortcut icon" href="_static/favicon.ico"/>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Driver" href="Driver.html" />
<link rel="prev" title="Development" href="development.html" />
<style type="text/css">
table.right { float: right; margin-left: 20px; }
table.right td { border: 1px solid #ccc; }
</style>
</head><body>
<div class="logo">
<a href="index.html"><img src="_static/logo.png" alt="LLVM Documentation"/></a>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="Driver.html" title="Driver"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="development.html" title="Development"
accesskey="P">previous</a> |</li>
<li><a href="index.html">lld Home</a> | </li>
<li class="nav-item nav-item-1"><a href="AtomLLD.html" >ATOM-based lld</a> »</li>
<li class="nav-item nav-item-2"><a href="development.html" accesskey="U">Development</a> »</li>
</ul>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Developing lld Readers</a><ul>
<li><a class="reference internal" href="#introduction">Introduction</a></li>
<li><a class="reference internal" href="#where-to-start">Where to start</a></li>
<li><a class="reference internal" href="#readers-are-factories">Readers are factories</a></li>
<li><a class="reference internal" href="#memory-ownership">Memory Ownership</a></li>
<li><a class="reference internal" href="#making-atoms">Making Atoms</a></li>
<li><a class="reference internal" href="#performance">Performance</a></li>
<li><a class="reference internal" href="#testing">Testing</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="development.html"
title="previous chapter">Development</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="Driver.html"
title="next chapter">Driver</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/Readers.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="developing-lld-readers">
<span id="readers"></span><h1>Developing lld Readers<a class="headerlink" href="#developing-lld-readers" title="Permalink to this headline">¶</a></h1>
<p>Note: this document discuss Mach-O port of LLD. For ELF and COFF,
see <a class="reference internal" href="index.html"><span class="doc">LLD - The LLVM Linker</span></a>.</p>
<div class="section" id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
<p>The purpose of a “Reader” is to take an object file in a particular format
and create an <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld::File</span></code> (which is a graph of Atoms)
representing the object file. A Reader inherits from
<code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld::Reader</span></code> which lives in
<code class="file docutils literal notranslate"><span class="pre">include/lld/Core/Reader.h</span></code> and
<code class="file docutils literal notranslate"><span class="pre">lib/Core/Reader.cpp</span></code>.</p>
<p>The Reader infrastructure for an object format <code class="docutils literal notranslate"><span class="pre">Foo</span></code> requires the
following pieces in order to fit into lld:</p>
<p><code class="file docutils literal notranslate"><span class="pre">include/lld/ReaderWriter/ReaderFoo.h</span></code></p>
<blockquote>
<div><dl class="class">
<dt id="_CPPv416ReaderOptionsFoo">
<span id="_CPPv316ReaderOptionsFoo"></span><span id="_CPPv216ReaderOptionsFoo"></span><span id="ReaderOptionsFoo"></span><em class="property">class </em><code class="descname">ReaderOptionsFoo</code> : <em class="property">public</em> ReaderOptions<a class="headerlink" href="#_CPPv416ReaderOptionsFoo" title="Permalink to this definition">¶</a><br /></dt>
<dd><p>This Options class is the only way to configure how the Reader will
parse any file into an <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld::Reader</span></code> object. This class
should be declared in the <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld</span></code> namespace.</p>
</dd></dl>
<dl class="function">
<dt id="_CPPv415createReaderFooR16ReaderOptionsFoo">
<span id="_CPPv315createReaderFooR16ReaderOptionsFoo"></span><span id="_CPPv215createReaderFooR16ReaderOptionsFoo"></span><span id="createReaderFoo__ReaderOptionsFooR"></span>Reader *<code class="descname">createReaderFoo</code><span class="sig-paren">(</span><a class="reference internal" href="#_CPPv416ReaderOptionsFoo" title="ReaderOptionsFoo">ReaderOptionsFoo</a> &<em>reader</em><span class="sig-paren">)</span><a class="headerlink" href="#_CPPv415createReaderFooR16ReaderOptionsFoo" title="Permalink to this definition">¶</a><br /></dt>
<dd><p>This factory function configures and create the Reader. This function
should be declared in the <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld</span></code> namespace.</p>
</dd></dl>
</div></blockquote>
<p><code class="file docutils literal notranslate"><span class="pre">lib/ReaderWriter/Foo/ReaderFoo.cpp</span></code></p>
<blockquote>
<div><dl class="class">
<dt id="_CPPv49ReaderFoo">
<span id="_CPPv39ReaderFoo"></span><span id="_CPPv29ReaderFoo"></span><span id="ReaderFoo"></span><em class="property">class </em><code class="descname">ReaderFoo</code> : <em class="property">public</em> Reader<a class="headerlink" href="#_CPPv49ReaderFoo" title="Permalink to this definition">¶</a><br /></dt>
<dd><p>This is the concrete Reader class which can be called to parse
object files. It should be declared in an anonymous namespace or
if there is shared code with the <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld::WriterFoo</span></code> you
can make a nested namespace (e.g. <code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">lld::foo</span></code>).</p>
</dd></dl>
</div></blockquote>
<p>You may have noticed that <a class="reference internal" href="#_CPPv49ReaderFoo" title="ReaderFoo"><code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">ReaderFoo</span></code></a> is not declared in the
<code class="docutils literal notranslate"><span class="pre">.h</span></code> file. An important design aspect of lld is that all Readers are
created <em>only</em> through an object-format-specific
<a class="reference internal" href="#_CPPv415createReaderFooR16ReaderOptionsFoo" title="createReaderFoo"><code class="xref cpp cpp-func docutils literal notranslate"><span class="pre">createReaderFoo()</span></code></a> factory function. The creation of the Reader is
parametrized through a <a class="reference internal" href="#_CPPv416ReaderOptionsFoo" title="ReaderOptionsFoo"><code class="xref cpp cpp-class docutils literal notranslate"><span class="pre">ReaderOptionsFoo</span></code></a> class. This options
class is the one-and-only way to control how the Reader operates when
parsing an input file into an Atom graph. For instance, you may want the
Reader to only accept certain architectures. The options class can be
instantiated from command line options or be programmatically configured.</p>
</div>
<div class="section" id="where-to-start">
<h2>Where to start<a class="headerlink" href="#where-to-start" title="Permalink to this headline">¶</a></h2>
<p>The lld project already has a skeleton of source code for Readers for
<code class="docutils literal notranslate"><span class="pre">ELF</span></code>, <code class="docutils literal notranslate"><span class="pre">PECOFF</span></code>, <code class="docutils literal notranslate"><span class="pre">MachO</span></code>, and lld’s native <code class="docutils literal notranslate"><span class="pre">YAML</span></code> graph format.
If your file format is a variant of one of those, you should modify the
existing Reader to support your variant. This is done by customizing the Options
class for the Reader and making appropriate changes to the <code class="docutils literal notranslate"><span class="pre">.cpp</span></code> file to
interpret those options and act accordingly.</p>
<p>If your object file format is not a variant of any existing Reader, you’ll need
to create a new Reader subclass with the organization described above.</p>
</div>
<div class="section" id="readers-are-factories">
<h2>Readers are factories<a class="headerlink" href="#readers-are-factories" title="Permalink to this headline">¶</a></h2>
<p>The linker will usually only instantiate your Reader once. That one Reader will
have its loadFile() method called many times with different input files.
To support multithreaded linking, the Reader may be parsing multiple input
files in parallel. Therefore, there should be no parsing state in you Reader
object. Any parsing state should be in ivars of your File subclass or in
some temporary object.</p>
<p>The key function to implement in a reader is:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">virtual</span> <span class="n">error_code</span> <span class="n">loadFile</span><span class="p">(</span><span class="n">LinkerInput</span> <span class="o">&</span><span class="nb">input</span><span class="p">,</span>
<span class="n">std</span><span class="p">::</span><span class="n">vector</span><span class="o"><</span><span class="n">std</span><span class="p">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">File</span><span class="o">>></span> <span class="o">&</span><span class="n">result</span><span class="p">);</span>
</pre></div>
</div>
<p>It takes a memory buffer (which contains the contents of the object file
being read) and returns an instantiated lld::File object which is
a collection of Atoms. The result is a vector of File pointers (instead of
simple a File pointer) because some file formats allow multiple object
“files” to be encoded in one file system file.</p>
</div>
<div class="section" id="memory-ownership">
<h2>Memory Ownership<a class="headerlink" href="#memory-ownership" title="Permalink to this headline">¶</a></h2>
<p>Atoms are always owned by their File object. During core linking when Atoms
are coalesced or stripped away, core linking does not delete them.
Core linking just removes those unused Atoms from its internal list.
The destructor of a File object is responsible for deleting all Atoms it
owns, and if ownership of the MemoryBuffer was passed to it, the File
destructor needs to delete that too.</p>
</div>
<div class="section" id="making-atoms">
<h2>Making Atoms<a class="headerlink" href="#making-atoms" title="Permalink to this headline">¶</a></h2>
<p>The internal model of lld is purely Atom based. But most object files do not
have an explicit concept of Atoms, instead most have “sections”. The way
to think of this is that a section is just a list of Atoms with common
attributes.</p>
<p>The first step in parsing section-based object files is to cleave each
section into a list of Atoms. The technique may vary by section type. For
code sections (e.g. .text), there are usually symbols at the start of each
function. Those symbol addresses are the points at which the section is
cleaved into discrete Atoms. Some file formats (like ELF) also include the
length of each symbol in the symbol table. Otherwise, the length of each
Atom is calculated to run to the start of the next symbol or the end of the
section.</p>
<p>Other sections types can be implicitly cleaved. For instance c-string literals
or unwind info (e.g. .eh_frame) can be cleaved by having the Reader look at
the content of the section. It is important to cleave sections into Atoms
to remove false dependencies. For instance the .eh_frame section often
has no symbols, but contains “pointers” to the functions for which it
has unwind info. If the .eh_frame section was not cleaved (but left as one
big Atom), there would always be a reference (from the eh_frame Atom) to
each function. So the linker would be unable to coalesce or dead stripped
away the function atoms.</p>
<p>The lld Atom model also requires that a reference to an undefined symbol be
modeled as a Reference to an UndefinedAtom. So the Reader also needs to
create an UndefinedAtom for each undefined symbol in the object file.</p>
<p>Once all Atoms have been created, the second step is to create References
(recall that Atoms are “nodes” and References are “edges”). Most References
are created by looking at the “relocation records” in the object file. If
a function contains a call to “malloc”, there is usually a relocation record
specifying the address in the section and the symbol table index. Your
Reader will need to convert the address to an Atom and offset and the symbol
table index into a target Atom. If “malloc” is not defined in the object file,
the target Atom of the Reference will be an UndefinedAtom.</p>
</div>
<div class="section" id="performance">
<h2>Performance<a class="headerlink" href="#performance" title="Permalink to this headline">¶</a></h2>
<p>Once you have the above working to parse an object file into Atoms and
References, you’ll want to look at performance. Some techniques that can
help performance are:</p>
<ul class="simple">
<li>Use llvm::BumpPtrAllocator or pre-allocate one big vector<Reference> and then
just have each atom point to its subrange of References in that vector.
This can be faster that allocating each Reference as separate object.</li>
<li>Pre-scan the symbol table and determine how many atoms are in each section
then allocate space for all the Atom objects at once.</li>
<li>Don’t copy symbol names or section content to each Atom, instead use
StringRef and ArrayRef in each Atom to point to its name and content in the
MemoryBuffer.</li>
</ul>
</div>
<div class="section" id="testing">
<h2>Testing<a class="headerlink" href="#testing" title="Permalink to this headline">¶</a></h2>
<p>We are still working on infrastructure to test Readers. The issue is that
you don’t want to check in binary files to the test suite. And the tools
for creating your object file from assembly source may not be available on
every OS.</p>
<p>We are investigating a way to use YAML to describe the section, symbols,
and content of a file. Then have some code which will write out an object
file from that YAML description.</p>
<p>Once that is in place, you can write test cases that contain section/symbols
YAML and is run through the linker to produce Atom/References based YAML which
is then run through FileCheck to verify the Atoms and References are as
expected.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="Driver.html" title="Driver"
>next</a> |</li>
<li class="right" >
<a href="development.html" title="Development"
>previous</a> |</li>
<li><a href="index.html">lld Home</a> | </li>
<li class="nav-item nav-item-1"><a href="AtomLLD.html" >ATOM-based lld</a> »</li>
<li class="nav-item nav-item-2"><a href="development.html" >Development</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2011-2020, LLVM Project.
Last updated on 2020-02-14.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.8.5.
</div>
</body>
</html>