<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>src/bootstrap/README.md - fork/rust @ 1.42.0</title>
<meta name="description" content="src/bootstrap/README.md in fork/rust at 1.42.0.">
<meta property="og:title" content="src/bootstrap/README.md - fork/rust @ 1.42.0">
<meta property="og:description" content="src/bootstrap/README.md in fork/rust at 1.42.0.">
<meta property="og:type" content="website">
<meta property="og:site_name" content="git.julia.blue">
<link rel="stylesheet" href="/css/style.css?v=62">
<script type="module" src="/js/main.js?v=62"></script>
</head>
<body>
<main data-ref="1.42.0" data-tip="b8cedc00407a4c56a3bda1ed605c6fc166655447">
<noscript>
  <div class="callout">
    <p>
      hey! this is a static git repo viewer. commit diffs, file history,
      older commits and other revisions won't load without javascript.
    </p>
    <p>
      you are still able to view source files on branch/tag tips,
      but the experience is degraded.
    </p>
  </div>
</noscript>
<header class="repo"><div>
<h1>fork/rust</h1>
</div>
<pre class="clone">git clone https://git.julia.blue/fork/rust</pre>
</header>
<p class="snapshot-commit"><span class="commit-author">bors</span><span class="commit-message" data-commit="b8cedc00407a4c56a3bda1ed605c6fc166655447">Auto merge of #69854 - pietroalbini:stable-next, r=Centril</span><span class="snapshot-sha sha" data-commit="b8cedc00407a4c56a3bda1ed605c6fc166655447">b8cedc00407</span><time datetime="2020-03-09T22:09:51+00:00" title="2020-03-09 22:09:51 +0000">2020-03-09</time></p>
<header class="topbar"><details class="switcher"><summary>1.42.0</summary><div class="menu"><strong>branches</strong><a href="/fork/rust/ref/master/">master</a><strong>tags</strong><a href="/fork/rust/ref/1.39.0/">1.39.0</a><a href="/fork/rust/ref/1.40.0/">1.40.0</a><a href="/fork/rust/ref/1.41.0/">1.41.0</a><a href="/fork/rust/ref/1.41.1/">1.41.1</a><a class="current" href="/fork/rust/ref/1.42.0/">1.42.0</a><a href="/fork/rust/ref/1.43.0/">1.43.0</a><a href="/fork/rust/ref/1.43.1/">1.43.1</a><a href="/fork/rust/ref/1.44.0/">1.44.0</a><a class="all" href="/fork/rust/refs">all refs →</a></div></details><nav class="crumbs"><a href="/fork/rust/">rust</a> / <a href="/fork/rust/ref/1.42.0/tree/src/">src</a> / <a href="/fork/rust/ref/1.42.0/tree/src/bootstrap/">bootstrap</a> / <span class="cur">README.md</span></nav>
<span class="view-stats"><span>12.7 KiB</span><span>335 lines</span></span><span class="actions"><a class="raw" href="/fork/rust/raw/c501378bff5491b0bba6cb39a3738f7e7f5845f4/src/bootstrap/README.md">raw</a></span></header>
<div class="markdown-view"><input type="radio" name="markdown-view" id="markdown-rendered" checked><label for="markdown-rendered">rendered</label><input type="radio" name="markdown-view" id="markdown-code"><label for="markdown-code">code</label><section class="readme rendered"><h1>rustbuild - Bootstrapping Rust</h1>
<p>This is an in-progress README which is targeted at helping to explain how Rust
is bootstrapped and in general some of the technical details of the build
system.</p>
<h2>Using rustbuild</h2>
<p>The rustbuild build system has a primary entry point, a top level <code>x.py</code> script:</p>
<pre class="language-bash"><a-o>$</a-o><a-f> </a-f><a-pr>python</a-pr> ./x.py build</pre>
<p>Note that if you're on Unix you should be able to execute the script directly:</p>
<pre class="language-bash"><a-o>$</a-o> <a-f>./x.py</a-f> build</pre>
<p>The script accepts commands, flags, and arguments to determine what to do:</p>
<ul>
<li>
<p><code>build</code> - a general purpose command for compiling code. Alone <code>build</code> will
bootstrap the entire compiler, and otherwise arguments passed indicate what to
build. For example:</p>
<pre># build the whole compiler
./x.py build

# build the stage1 compiler
./x.py build --stage 1

# build stage0 libstd
./x.py build --stage 0 src/libstd

# build a particular crate in stage0
./x.py build --stage 0 src/libtest
</pre>
<p>If files are dirty that would normally be rebuilt from stage 0, that can be
overridden using <code>--keep-stage 0</code>. Using <code>--keep-stage n</code> will skip all steps
that belong to stage n or earlier:</p>
<pre># keep old build products for stage 0 and build stage 1
./x.py build --keep-stage 0 --stage 1
</pre>
</li>
<li>
<p><code>test</code> - a command for executing unit tests. Like the <code>build</code> command this
will execute the entire test suite by default, and otherwise it can be used to
select which test suite is run:</p>
<pre># run all unit tests
./x.py test

# execute the UI test suite
./x.py test src/test/ui

# execute only some tests in the UI test suite
./x.py test src/test/ui --test-args substring-of-test-name

# execute tests in the standard library in stage0
./x.py test --stage 0 src/libstd

# execute tests in the core and standard library in stage0,
# without running doc tests (thus avoid depending on building the compiler)
./x.py test --stage 0 --no-doc src/libcore src/libstd

# execute all doc tests
./x.py test src/doc
</pre>
</li>
<li>
<p><code>doc</code> - a command for building documentation. Like above can take arguments
for what to document.</p>
</li>
</ul>
<h2>Configuring rustbuild</h2>
<p>There are currently two methods for configuring the rustbuild build system.</p>
<p>First, rustbuild offers a TOML-based configuration system with a <code>config.toml</code>
file. An example of this configuration can be found at <code>config.toml.example</code>,
and the configuration file can also be passed as <code>--config path/to/config.toml</code>
if the build system is being invoked manually (via the python script).</p>
<p>Next, the <code>./configure</code> options serialized in <code>config.mk</code> will be
parsed and read. That is, if any <code>./configure</code> options are passed, they'll be
handled naturally. <code>./configure</code> should almost never be used for local
installations, and is primarily useful for CI. Prefer to customize behavior
using <code>config.toml</code>.</p>
<p>Finally, rustbuild makes use of the <a href="https://github.com/alexcrichton/gcc-rs">gcc-rs crate</a> which has <a href="https://github.com/alexcrichton/gcc-rs#external-configuration-via-environment-variables">its own
method</a> of configuring C compilers and C flags via environment
variables.</p>
<h2>Build stages</h2>
<p>The rustbuild build system goes through a few phases to actually build the
compiler. What actually happens when you invoke rustbuild is:</p>
<ol>
<li>The entry point script, <code>x.py</code> is run. This script is
responsible for downloading the stage0 compiler/Cargo binaries, and it then
compiles the build system itself (this folder). Finally, it then invokes the
actual <code>bootstrap</code> binary build system.</li>
<li>In Rust, <code>bootstrap</code> will slurp up all configuration, perform a number of
sanity checks (compilers exist for example), and then start building the
stage0 artifacts.</li>
<li>The stage0 <code>cargo</code> downloaded earlier is used to build the standard library
and the compiler, and then these binaries are then copied to the <code>stage1</code>
directory. That compiler is then used to generate the stage1 artifacts which
are then copied to the stage2 directory, and then finally the stage2
artifacts are generated using that compiler.</li>
</ol>
<p>The goal of each stage is to (a) leverage Cargo as much as possible and failing
that (b) leverage Rust as much as possible!</p>
<h2>Incremental builds</h2>
<p>You can configure rustbuild to use incremental compilation. Because
incremental is new and evolving rapidly, if you want to use it, it is
recommended that you replace the snapshot with a locally installed
nightly build of rustc. You will want to keep this up to date.</p>
<p>To follow this course of action, first thing you will want to do is to
install a nightly, presumably using <code>rustup</code>. You will then want to
configure your directory to use this build, like so:</p>
<pre class="language-bash"><a-c># configure to use local rust instead of downloading a beta.</a-c>
<a-c># `--local-rust-root` is optional here. If elided, we will</a-c>
<a-c># use whatever rustc we find on your PATH.</a-c>
<a-o>$</a-o> <a-f>./configure</a-f> <a-co>--local-rust-root=~/.cargo/</a-co> <a-co>--enable-local-rebuild</a-co></pre>
<p>After that, you can use the <code>--incremental</code> flag to actually do
incremental builds:</p>
<pre class="language-bash"><a-o>$</a-o> <a-f>./x.py</a-f> build <a-co>--incremental</a-co></pre>
<p>The <code>--incremental</code> flag will store incremental compilation artifacts
in <code>build/&lt;host&gt;/stage0-incremental</code>. Note that we only use incremental
compilation for the stage0 -&gt; stage1 compilation -- this is because
the stage1 compiler is changing, and we don't try to cache and reuse
incremental artifacts across different versions of the compiler. For
this reason, <code>--incremental</code> defaults to <code>--stage 1</code> (though you can
manually select a higher stage, if you prefer).</p>
<p>You can always drop the <code>--incremental</code> to build as normal (but you
will still be using the local nightly as your bootstrap).</p>
<h2>Directory Layout</h2>
<p>This build system houses all output under the <code>build</code> directory, which looks
like this:</p>
<pre class="language-bash"><a-c># Root folder of all output. Everything is scoped underneath here</a-c>
<a-f>build/</a-f>

  <a-c># Location where the stage0 compiler downloads are all cached. This directory</a-c>
  <a-c># only contains the tarballs themselves as they&#39;re extracted elsewhere.</a-c>
  <a-f>cache/</a-f>
    <a-f>2015-12-19/</a-f>
    <a-f>2016-01-15/</a-f>
    <a-f>2016-01-21/</a-f>
    <a-f>...</a-f>

  <a-c># Output directory for building this build system itself. The stage0</a-c>
  <a-c># cargo/rustc are used to build the build system into this location.</a-c>
  <a-f>bootstrap/</a-f>
    <a-f>debug/</a-f>
    <a-f>release/</a-f>

  <a-c># Output of the dist-related steps like dist-std, dist-rustc, and dist-docs</a-c>
  <a-f>dist/</a-f>

  <a-c># Temporary directory used for various input/output as part of various stages</a-c>
  <a-f>tmp/</a-f>

  <a-c># Each remaining directory is scoped by the &quot;host&quot; triple of compilation at</a-c>
  <a-c># hand.</a-c>
  <a-f>x86_64-unknown-linux-gnu/</a-f>

    <a-c># The build artifacts for the `compiler-rt` library for the target this</a-c>
    <a-c># folder is under. The exact layout here will likely depend on the platform,</a-c>
    <a-c># and this is also built with CMake so the build system is also likely</a-c>
    <a-c># different.</a-c>
    <a-f>compiler-rt/</a-f>
      <a-f>build/</a-f>

    <a-c># Output folder for LLVM if it is compiled for this target</a-c>
    <a-f>llvm/</a-f>

      <a-c># build folder (e.g. the platform-specific build system). Like with</a-c>
      <a-c># compiler-rt this is compiled with CMake</a-c>
      <a-f>build/</a-f>

      <a-c># Installation of LLVM. Note that we run the equivalent of &#39;make install&#39;</a-c>
      <a-c># for LLVM to setup these folders.</a-c>
      <a-f>bin/</a-f>
      <a-f>lib/</a-f>
      <a-f>include/</a-f>
      <a-f>share/</a-f>
      <a-f>...</a-f>

    <a-c># Output folder for all documentation of this target. This is what&#39;s filled</a-c>
    <a-c># in whenever the `doc` step is run.</a-c>
    <a-f>doc/</a-f>

    <a-c># Output for all compiletest-based test suites</a-c>
    <a-f>test/</a-f>
      <a-f>ui/</a-f>
      <a-f>compile-fail/</a-f>
      <a-f>debuginfo/</a-f>
      <a-f>...</a-f>

    <a-c># Location where the stage0 Cargo and Rust compiler are unpacked. This</a-c>
    <a-c># directory is purely an extracted and overlaid tarball of these two (done</a-c>
    <a-c># by the bootstrapy python script). In theory the build system does not</a-c>
    <a-c># modify anything under this directory afterwards.</a-c>
    <a-f>stage0/</a-f>

    <a-c># These to build directories are the cargo output directories for builds of</a-c>
    <a-c># the standard library and compiler, respectively. Internally these may also</a-c>
    <a-c># have other target directories, which represent artifacts being compiled</a-c>
    <a-c># from the host to the specified target.</a-c>
    <a-c>#</a-c>
    <a-c># Essentially, each of these directories is filled in by one `cargo`</a-c>
    <a-c># invocation. The build system instruments calling Cargo in the right order</a-c>
    <a-c># with the right variables to ensure these are filled in correctly.</a-c>
    <a-f>stageN-std/</a-f>
    <a-f>stageN-test/</a-f>
    <a-f>stageN-rustc/</a-f>
    <a-f>stageN-tools/</a-f>

    <a-c># This is a special case of the above directories, **not** filled in via</a-c>
    <a-c># Cargo but rather the build system itself. The stage0 compiler already has</a-c>
    <a-c># a set of target libraries for its own host triple (in its own sysroot)</a-c>
    <a-c># inside of stage0/. When we run the stage0 compiler to bootstrap more</a-c>
    <a-c># things, however, we don&#39;t want to use any of these libraries (as those are</a-c>
    <a-c># the ones that we&#39;re building). So essentially, when the stage1 compiler is</a-c>
    <a-c># being compiled (e.g. after libstd has been built), *this* is used as the</a-c>
    <a-c># sysroot for the stage0 compiler being run.</a-c>
    <a-c>#</a-c>
    <a-c># Basically this directory is just a temporary artifact use to configure the</a-c>
    <a-c># stage0 compiler to ensure that the libstd we just built is used to</a-c>
    <a-c># compile the stage1 compiler.</a-c>
    <a-f>stage0-sysroot/lib/</a-f>

    <a-c># These output directories are intended to be standalone working</a-c>
    <a-c># implementations of the compiler (corresponding to each stage). The build</a-c>
    <a-c># system will link (using hard links) output from stageN-{std,rustc} into</a-c>
    <a-c># each of these directories.</a-c>
    <a-c>#</a-c>
    <a-c># In theory there is no extra build output in these directories.</a-c>
    <a-f>stage1/</a-f>
    <a-f>stage2/</a-f>
    <a-f>stage3/</a-f></pre>
<h2>Cargo projects</h2>
<p>The current build is unfortunately not quite as simple as <code>cargo build</code> in a
directory, but rather the compiler is split into three different Cargo projects:</p>
<ul>
<li><code>src/libstd</code> - the standard library</li>
<li><code>src/libtest</code> - testing support, depends on libstd</li>
<li><code>src/rustc</code> - the actual compiler itself</li>
</ul>
<p>Each "project" has a corresponding Cargo.lock file with all dependencies, and
this means that building the compiler involves running Cargo three times. The
structure here serves two goals:</p>
<ol>
<li>Facilitating dependencies coming from crates.io. These dependencies don't
depend on <code>std</code>, so libstd is a separate project compiled ahead of time
before the actual compiler builds.</li>
<li>Splitting "host artifacts" from "target artifacts". That is, when building
code for an arbitrary target you don't need the entire compiler, but you'll
end up needing libraries like libtest that depend on std but also want to use
crates.io dependencies. Hence, libtest is split out as its own project that
is sequenced after <code>std</code> but before <code>rustc</code>. This project is built for all
targets.</li>
</ol>
<p>There is some loss in build parallelism here because libtest can be compiled in
parallel with a number of rustc artifacts, but in theory the loss isn't too bad!</p>
<h2>Build tools</h2>
<p>We've actually got quite a few tools that we use in the compiler's build system
and for testing. To organize these, each tool is a project in <code>src/tools</code> with a
corresponding <code>Cargo.toml</code>. All tools are compiled with Cargo (currently having
independent <code>Cargo.lock</code> files) and do not currently explicitly depend on the
compiler or standard library. Compiling each tool is sequenced after the
appropriate libstd/libtest/librustc compile above.</p>
<h2>Extending rustbuild</h2>
<p>So you'd like to add a feature to the rustbuild build system or just fix a bug.
Great! One of the major motivational factors for moving away from <code>make</code> is that
Rust is in theory much easier to read, modify, and write. If you find anything
excessively confusing, please open an issue on this and we'll try to get it
documented or simplified pronto.</p>
<p>First up, you'll probably want to read over the documentation above as that'll
give you a high level overview of what rustbuild is doing. You also probably
want to play around a bit yourself by just getting it up and running before you
dive too much into the actual build system itself.</p>
<p>After that, each module in rustbuild should have enough documentation to keep
you up and running. Some general areas that you may be interested in modifying
are:</p>
<ul>
<li>Adding a new build tool? Take a look at <code>bootstrap/tool.rs</code> for examples of
other tools.</li>
<li>Adding a new compiler crate? Look no further! Adding crates can be done by
adding a new directory with <code>Cargo.toml</code> followed by configuring all
<code>Cargo.toml</code> files accordingly.</li>
<li>Adding a new dependency from crates.io? This should just work inside the
compiler artifacts stage (everything other than libtest and libstd).</li>
<li>Adding a new configuration option? You'll want to modify <code>bootstrap/flags.rs</code>
for command line flags and then <code>bootstrap/config.rs</code> to copy the flags to the
<code>Config</code> struct.</li>
<li>Adding a sanity check? Take a look at <code>bootstrap/sanity.rs</code>.</li>
</ul>
<p>If you have any questions feel free to reach out on <code>#infra</code> channel in the
<a href="https://discord.gg/rust-lang">Rust Discord server</a> or ask on internals.rust-lang.org. When
you encounter bugs, please file issues on the rust-lang/rust issue tracker.</p>
</section><div class="code-panel"><pre class="code src" data-hl="src/bootstrap/README.md"><span class="code-line"><a class="ln" id="L1" href="#L1">1</a><span class="code-text"># rustbuild - Bootstrapping Rust</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L2" href="#L2">2</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L3" href="#L3">3</a><span class="code-text">This is an in-progress README which is targeted at helping to explain how Rust</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L4" href="#L4">4</a><span class="code-text">is bootstrapped and in general some of the technical details of the build</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L5" href="#L5">5</a><span class="code-text">system.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L6" href="#L6">6</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L7" href="#L7">7</a><span class="code-text">## Using rustbuild</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L8" href="#L8">8</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L9" href="#L9">9</a><span class="code-text">The rustbuild build system has a primary entry point, a top level `x.py` script:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L10" href="#L10">10</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L11" href="#L11">11</a><span class="code-text">```sh</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L12" href="#L12">12</a><span class="code-text">$ python ./x.py build</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L13" href="#L13">13</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L14" href="#L14">14</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L15" href="#L15">15</a><span class="code-text">Note that if you're on Unix you should be able to execute the script directly:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L16" href="#L16">16</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L17" href="#L17">17</a><span class="code-text">```sh</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L18" href="#L18">18</a><span class="code-text">$ ./x.py build</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L19" href="#L19">19</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L20" href="#L20">20</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L21" href="#L21">21</a><span class="code-text">The script accepts commands, flags, and arguments to determine what to do:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L22" href="#L22">22</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L23" href="#L23">23</a><span class="code-text">* `build` - a general purpose command for compiling code. Alone `build` will</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L24" href="#L24">24</a><span class="code-text">  bootstrap the entire compiler, and otherwise arguments passed indicate what to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L25" href="#L25">25</a><span class="code-text">  build. For example:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L26" href="#L26">26</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L27" href="#L27">27</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L28" href="#L28">28</a><span class="code-text">  # build the whole compiler</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L29" href="#L29">29</a><span class="code-text">  ./x.py build</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L30" href="#L30">30</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L31" href="#L31">31</a><span class="code-text">  # build the stage1 compiler</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L32" href="#L32">32</a><span class="code-text">  ./x.py build --stage 1</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L33" href="#L33">33</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L34" href="#L34">34</a><span class="code-text">  # build stage0 libstd</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L35" href="#L35">35</a><span class="code-text">  ./x.py build --stage 0 src/libstd</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L36" href="#L36">36</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L37" href="#L37">37</a><span class="code-text">  # build a particular crate in stage0</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L38" href="#L38">38</a><span class="code-text">  ./x.py build --stage 0 src/libtest</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L39" href="#L39">39</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L40" href="#L40">40</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L41" href="#L41">41</a><span class="code-text">  If files are dirty that would normally be rebuilt from stage 0, that can be</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L42" href="#L42">42</a><span class="code-text">  overridden using `--keep-stage 0`. Using `--keep-stage n` will skip all steps</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L43" href="#L43">43</a><span class="code-text">  that belong to stage n or earlier:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L44" href="#L44">44</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L45" href="#L45">45</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L46" href="#L46">46</a><span class="code-text">  # keep old build products for stage 0 and build stage 1</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L47" href="#L47">47</a><span class="code-text">  ./x.py build --keep-stage 0 --stage 1</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L48" href="#L48">48</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L49" href="#L49">49</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L50" href="#L50">50</a><span class="code-text">* `test` - a command for executing unit tests. Like the `build` command this</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L51" href="#L51">51</a><span class="code-text">  will execute the entire test suite by default, and otherwise it can be used to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L52" href="#L52">52</a><span class="code-text">  select which test suite is run:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L53" href="#L53">53</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L54" href="#L54">54</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L55" href="#L55">55</a><span class="code-text">  # run all unit tests</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L56" href="#L56">56</a><span class="code-text">  ./x.py test</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L57" href="#L57">57</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L58" href="#L58">58</a><span class="code-text">  # execute the UI test suite</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L59" href="#L59">59</a><span class="code-text">  ./x.py test src/test/ui</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L60" href="#L60">60</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L61" href="#L61">61</a><span class="code-text">  # execute only some tests in the UI test suite</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L62" href="#L62">62</a><span class="code-text">  ./x.py test src/test/ui --test-args substring-of-test-name</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L63" href="#L63">63</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L64" href="#L64">64</a><span class="code-text">  # execute tests in the standard library in stage0</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L65" href="#L65">65</a><span class="code-text">  ./x.py test --stage 0 src/libstd</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L66" href="#L66">66</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L67" href="#L67">67</a><span class="code-text">  # execute tests in the core and standard library in stage0,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L68" href="#L68">68</a><span class="code-text">  # without running doc tests (thus avoid depending on building the compiler)</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L69" href="#L69">69</a><span class="code-text">  ./x.py test --stage 0 --no-doc src/libcore src/libstd</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L70" href="#L70">70</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L71" href="#L71">71</a><span class="code-text">  # execute all doc tests</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L72" href="#L72">72</a><span class="code-text">  ./x.py test src/doc</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L73" href="#L73">73</a><span class="code-text">  ```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L74" href="#L74">74</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L75" href="#L75">75</a><span class="code-text">* `doc` - a command for building documentation. Like above can take arguments</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L76" href="#L76">76</a><span class="code-text">  for what to document.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L77" href="#L77">77</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L78" href="#L78">78</a><span class="code-text">## Configuring rustbuild</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L79" href="#L79">79</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L80" href="#L80">80</a><span class="code-text">There are currently two methods for configuring the rustbuild build system.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L81" href="#L81">81</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L82" href="#L82">82</a><span class="code-text">First, rustbuild offers a TOML-based configuration system with a `config.toml`</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L83" href="#L83">83</a><span class="code-text">file. An example of this configuration can be found at `config.toml.example`,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L84" href="#L84">84</a><span class="code-text">and the configuration file can also be passed as `--config path/to/config.toml`</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L85" href="#L85">85</a><span class="code-text">if the build system is being invoked manually (via the python script).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L86" href="#L86">86</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L87" href="#L87">87</a><span class="code-text">Next, the `./configure` options serialized in `config.mk` will be</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L88" href="#L88">88</a><span class="code-text">parsed and read. That is, if any `./configure` options are passed, they'll be</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L89" href="#L89">89</a><span class="code-text">handled naturally. `./configure` should almost never be used for local</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L90" href="#L90">90</a><span class="code-text">installations, and is primarily useful for CI. Prefer to customize behavior</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L91" href="#L91">91</a><span class="code-text">using `config.toml`.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L92" href="#L92">92</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L93" href="#L93">93</a><span class="code-text">Finally, rustbuild makes use of the [gcc-rs crate] which has [its own</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L94" href="#L94">94</a><span class="code-text">method][env-vars] of configuring C compilers and C flags via environment</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L95" href="#L95">95</a><span class="code-text">variables.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L96" href="#L96">96</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L97" href="#L97">97</a><span class="code-text">[gcc-rs crate]: https://github.com/alexcrichton/gcc-rs</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L98" href="#L98">98</a><span class="code-text">[env-vars]: https://github.com/alexcrichton/gcc-rs#external-configuration-via-environment-variables</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L99" href="#L99">99</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L100" href="#L100">100</a><span class="code-text">## Build stages</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L101" href="#L101">101</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L102" href="#L102">102</a><span class="code-text">The rustbuild build system goes through a few phases to actually build the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L103" href="#L103">103</a><span class="code-text">compiler. What actually happens when you invoke rustbuild is:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L104" href="#L104">104</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L105" href="#L105">105</a><span class="code-text">1. The entry point script, `x.py` is run. This script is</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L106" href="#L106">106</a><span class="code-text">   responsible for downloading the stage0 compiler/Cargo binaries, and it then</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L107" href="#L107">107</a><span class="code-text">   compiles the build system itself (this folder). Finally, it then invokes the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L108" href="#L108">108</a><span class="code-text">   actual `bootstrap` binary build system.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L109" href="#L109">109</a><span class="code-text">2. In Rust, `bootstrap` will slurp up all configuration, perform a number of</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L110" href="#L110">110</a><span class="code-text">   sanity checks (compilers exist for example), and then start building the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L111" href="#L111">111</a><span class="code-text">   stage0 artifacts.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L112" href="#L112">112</a><span class="code-text">3. The stage0 `cargo` downloaded earlier is used to build the standard library</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L113" href="#L113">113</a><span class="code-text">   and the compiler, and then these binaries are then copied to the `stage1`</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L114" href="#L114">114</a><span class="code-text">   directory. That compiler is then used to generate the stage1 artifacts which</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L115" href="#L115">115</a><span class="code-text">   are then copied to the stage2 directory, and then finally the stage2</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L116" href="#L116">116</a><span class="code-text">   artifacts are generated using that compiler.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L117" href="#L117">117</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L118" href="#L118">118</a><span class="code-text">The goal of each stage is to (a) leverage Cargo as much as possible and failing</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L119" href="#L119">119</a><span class="code-text">that (b) leverage Rust as much as possible!</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L120" href="#L120">120</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L121" href="#L121">121</a><span class="code-text">## Incremental builds</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L122" href="#L122">122</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L123" href="#L123">123</a><span class="code-text">You can configure rustbuild to use incremental compilation. Because</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L124" href="#L124">124</a><span class="code-text">incremental is new and evolving rapidly, if you want to use it, it is</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L125" href="#L125">125</a><span class="code-text">recommended that you replace the snapshot with a locally installed</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L126" href="#L126">126</a><span class="code-text">nightly build of rustc. You will want to keep this up to date.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L127" href="#L127">127</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L128" href="#L128">128</a><span class="code-text">To follow this course of action, first thing you will want to do is to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L129" href="#L129">129</a><span class="code-text">install a nightly, presumably using `rustup`. You will then want to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L130" href="#L130">130</a><span class="code-text">configure your directory to use this build, like so:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L131" href="#L131">131</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L132" href="#L132">132</a><span class="code-text">```sh</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L133" href="#L133">133</a><span class="code-text"># configure to use local rust instead of downloading a beta.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L134" href="#L134">134</a><span class="code-text"># `--local-rust-root` is optional here. If elided, we will</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L135" href="#L135">135</a><span class="code-text"># use whatever rustc we find on your PATH.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L136" href="#L136">136</a><span class="code-text">$ ./configure --local-rust-root=~/.cargo/ --enable-local-rebuild</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L137" href="#L137">137</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L138" href="#L138">138</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L139" href="#L139">139</a><span class="code-text">After that, you can use the `--incremental` flag to actually do</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L140" href="#L140">140</a><span class="code-text">incremental builds:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L141" href="#L141">141</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L142" href="#L142">142</a><span class="code-text">```sh</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L143" href="#L143">143</a><span class="code-text">$ ./x.py build --incremental</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L144" href="#L144">144</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L145" href="#L145">145</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L146" href="#L146">146</a><span class="code-text">The `--incremental` flag will store incremental compilation artifacts</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L147" href="#L147">147</a><span class="code-text">in `build/&lt;host&gt;/stage0-incremental`. Note that we only use incremental</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L148" href="#L148">148</a><span class="code-text">compilation for the stage0 -&gt; stage1 compilation -- this is because</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L149" href="#L149">149</a><span class="code-text">the stage1 compiler is changing, and we don't try to cache and reuse</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L150" href="#L150">150</a><span class="code-text">incremental artifacts across different versions of the compiler. For</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L151" href="#L151">151</a><span class="code-text">this reason, `--incremental` defaults to `--stage 1` (though you can</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L152" href="#L152">152</a><span class="code-text">manually select a higher stage, if you prefer).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L153" href="#L153">153</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L154" href="#L154">154</a><span class="code-text">You can always drop the `--incremental` to build as normal (but you</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L155" href="#L155">155</a><span class="code-text">will still be using the local nightly as your bootstrap).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L156" href="#L156">156</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L157" href="#L157">157</a><span class="code-text">## Directory Layout</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L158" href="#L158">158</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L159" href="#L159">159</a><span class="code-text">This build system houses all output under the `build` directory, which looks</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L160" href="#L160">160</a><span class="code-text">like this:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L161" href="#L161">161</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L162" href="#L162">162</a><span class="code-text">```sh</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L163" href="#L163">163</a><span class="code-text"># Root folder of all output. Everything is scoped underneath here</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L164" href="#L164">164</a><span class="code-text">build/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L165" href="#L165">165</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L166" href="#L166">166</a><span class="code-text">  # Location where the stage0 compiler downloads are all cached. This directory</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L167" href="#L167">167</a><span class="code-text">  # only contains the tarballs themselves as they're extracted elsewhere.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L168" href="#L168">168</a><span class="code-text">  cache/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L169" href="#L169">169</a><span class="code-text">    2015-12-19/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L170" href="#L170">170</a><span class="code-text">    2016-01-15/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L171" href="#L171">171</a><span class="code-text">    2016-01-21/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L172" href="#L172">172</a><span class="code-text">    ...</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L173" href="#L173">173</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L174" href="#L174">174</a><span class="code-text">  # Output directory for building this build system itself. The stage0</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L175" href="#L175">175</a><span class="code-text">  # cargo/rustc are used to build the build system into this location.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L176" href="#L176">176</a><span class="code-text">  bootstrap/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L177" href="#L177">177</a><span class="code-text">    debug/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L178" href="#L178">178</a><span class="code-text">    release/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L179" href="#L179">179</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L180" href="#L180">180</a><span class="code-text">  # Output of the dist-related steps like dist-std, dist-rustc, and dist-docs</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L181" href="#L181">181</a><span class="code-text">  dist/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L182" href="#L182">182</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L183" href="#L183">183</a><span class="code-text">  # Temporary directory used for various input/output as part of various stages</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L184" href="#L184">184</a><span class="code-text">  tmp/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L185" href="#L185">185</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L186" href="#L186">186</a><span class="code-text">  # Each remaining directory is scoped by the &quot;host&quot; triple of compilation at</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L187" href="#L187">187</a><span class="code-text">  # hand.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L188" href="#L188">188</a><span class="code-text">  x86_64-unknown-linux-gnu/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L189" href="#L189">189</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L190" href="#L190">190</a><span class="code-text">    # The build artifacts for the `compiler-rt` library for the target this</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L191" href="#L191">191</a><span class="code-text">    # folder is under. The exact layout here will likely depend on the platform,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L192" href="#L192">192</a><span class="code-text">    # and this is also built with CMake so the build system is also likely</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L193" href="#L193">193</a><span class="code-text">    # different.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L194" href="#L194">194</a><span class="code-text">    compiler-rt/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L195" href="#L195">195</a><span class="code-text">      build/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L196" href="#L196">196</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L197" href="#L197">197</a><span class="code-text">    # Output folder for LLVM if it is compiled for this target</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L198" href="#L198">198</a><span class="code-text">    llvm/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L199" href="#L199">199</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L200" href="#L200">200</a><span class="code-text">      # build folder (e.g. the platform-specific build system). Like with</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L201" href="#L201">201</a><span class="code-text">      # compiler-rt this is compiled with CMake</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L202" href="#L202">202</a><span class="code-text">      build/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L203" href="#L203">203</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L204" href="#L204">204</a><span class="code-text">      # Installation of LLVM. Note that we run the equivalent of 'make install'</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L205" href="#L205">205</a><span class="code-text">      # for LLVM to setup these folders.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L206" href="#L206">206</a><span class="code-text">      bin/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L207" href="#L207">207</a><span class="code-text">      lib/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L208" href="#L208">208</a><span class="code-text">      include/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L209" href="#L209">209</a><span class="code-text">      share/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L210" href="#L210">210</a><span class="code-text">      ...</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L211" href="#L211">211</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L212" href="#L212">212</a><span class="code-text">    # Output folder for all documentation of this target. This is what's filled</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L213" href="#L213">213</a><span class="code-text">    # in whenever the `doc` step is run.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L214" href="#L214">214</a><span class="code-text">    doc/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L215" href="#L215">215</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L216" href="#L216">216</a><span class="code-text">    # Output for all compiletest-based test suites</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L217" href="#L217">217</a><span class="code-text">    test/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L218" href="#L218">218</a><span class="code-text">      ui/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L219" href="#L219">219</a><span class="code-text">      compile-fail/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L220" href="#L220">220</a><span class="code-text">      debuginfo/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L221" href="#L221">221</a><span class="code-text">      ...</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L222" href="#L222">222</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L223" href="#L223">223</a><span class="code-text">    # Location where the stage0 Cargo and Rust compiler are unpacked. This</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L224" href="#L224">224</a><span class="code-text">    # directory is purely an extracted and overlaid tarball of these two (done</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L225" href="#L225">225</a><span class="code-text">    # by the bootstrapy python script). In theory the build system does not</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L226" href="#L226">226</a><span class="code-text">    # modify anything under this directory afterwards.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L227" href="#L227">227</a><span class="code-text">    stage0/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L228" href="#L228">228</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L229" href="#L229">229</a><span class="code-text">    # These to build directories are the cargo output directories for builds of</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L230" href="#L230">230</a><span class="code-text">    # the standard library and compiler, respectively. Internally these may also</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L231" href="#L231">231</a><span class="code-text">    # have other target directories, which represent artifacts being compiled</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L232" href="#L232">232</a><span class="code-text">    # from the host to the specified target.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L233" href="#L233">233</a><span class="code-text">    #</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L234" href="#L234">234</a><span class="code-text">    # Essentially, each of these directories is filled in by one `cargo`</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L235" href="#L235">235</a><span class="code-text">    # invocation. The build system instruments calling Cargo in the right order</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L236" href="#L236">236</a><span class="code-text">    # with the right variables to ensure these are filled in correctly.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L237" href="#L237">237</a><span class="code-text">    stageN-std/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L238" href="#L238">238</a><span class="code-text">    stageN-test/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L239" href="#L239">239</a><span class="code-text">    stageN-rustc/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L240" href="#L240">240</a><span class="code-text">    stageN-tools/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L241" href="#L241">241</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L242" href="#L242">242</a><span class="code-text">    # This is a special case of the above directories, **not** filled in via</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L243" href="#L243">243</a><span class="code-text">    # Cargo but rather the build system itself. The stage0 compiler already has</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L244" href="#L244">244</a><span class="code-text">    # a set of target libraries for its own host triple (in its own sysroot)</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L245" href="#L245">245</a><span class="code-text">    # inside of stage0/. When we run the stage0 compiler to bootstrap more</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L246" href="#L246">246</a><span class="code-text">    # things, however, we don't want to use any of these libraries (as those are</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L247" href="#L247">247</a><span class="code-text">    # the ones that we're building). So essentially, when the stage1 compiler is</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L248" href="#L248">248</a><span class="code-text">    # being compiled (e.g. after libstd has been built), *this* is used as the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L249" href="#L249">249</a><span class="code-text">    # sysroot for the stage0 compiler being run.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L250" href="#L250">250</a><span class="code-text">    #</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L251" href="#L251">251</a><span class="code-text">    # Basically this directory is just a temporary artifact use to configure the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L252" href="#L252">252</a><span class="code-text">    # stage0 compiler to ensure that the libstd we just built is used to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L253" href="#L253">253</a><span class="code-text">    # compile the stage1 compiler.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L254" href="#L254">254</a><span class="code-text">    stage0-sysroot/lib/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L255" href="#L255">255</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L256" href="#L256">256</a><span class="code-text">    # These output directories are intended to be standalone working</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L257" href="#L257">257</a><span class="code-text">    # implementations of the compiler (corresponding to each stage). The build</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L258" href="#L258">258</a><span class="code-text">    # system will link (using hard links) output from stageN-{std,rustc} into</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L259" href="#L259">259</a><span class="code-text">    # each of these directories.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L260" href="#L260">260</a><span class="code-text">    #</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L261" href="#L261">261</a><span class="code-text">    # In theory there is no extra build output in these directories.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L262" href="#L262">262</a><span class="code-text">    stage1/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L263" href="#L263">263</a><span class="code-text">    stage2/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L264" href="#L264">264</a><span class="code-text">    stage3/</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L265" href="#L265">265</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L266" href="#L266">266</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L267" href="#L267">267</a><span class="code-text">## Cargo projects</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L268" href="#L268">268</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L269" href="#L269">269</a><span class="code-text">The current build is unfortunately not quite as simple as `cargo build` in a</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L270" href="#L270">270</a><span class="code-text">directory, but rather the compiler is split into three different Cargo projects:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L271" href="#L271">271</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L272" href="#L272">272</a><span class="code-text">* `src/libstd` - the standard library</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L273" href="#L273">273</a><span class="code-text">* `src/libtest` - testing support, depends on libstd</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L274" href="#L274">274</a><span class="code-text">* `src/rustc` - the actual compiler itself</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L275" href="#L275">275</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L276" href="#L276">276</a><span class="code-text">Each &quot;project&quot; has a corresponding Cargo.lock file with all dependencies, and</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L277" href="#L277">277</a><span class="code-text">this means that building the compiler involves running Cargo three times. The</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L278" href="#L278">278</a><span class="code-text">structure here serves two goals:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L279" href="#L279">279</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L280" href="#L280">280</a><span class="code-text">1. Facilitating dependencies coming from crates.io. These dependencies don't</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L281" href="#L281">281</a><span class="code-text">   depend on `std`, so libstd is a separate project compiled ahead of time</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L282" href="#L282">282</a><span class="code-text">   before the actual compiler builds.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L283" href="#L283">283</a><span class="code-text">2. Splitting &quot;host artifacts&quot; from &quot;target artifacts&quot;. That is, when building</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L284" href="#L284">284</a><span class="code-text">   code for an arbitrary target you don't need the entire compiler, but you'll</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L285" href="#L285">285</a><span class="code-text">   end up needing libraries like libtest that depend on std but also want to use</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L286" href="#L286">286</a><span class="code-text">   crates.io dependencies. Hence, libtest is split out as its own project that</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L287" href="#L287">287</a><span class="code-text">   is sequenced after `std` but before `rustc`. This project is built for all</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L288" href="#L288">288</a><span class="code-text">   targets.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L289" href="#L289">289</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L290" href="#L290">290</a><span class="code-text">There is some loss in build parallelism here because libtest can be compiled in</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L291" href="#L291">291</a><span class="code-text">parallel with a number of rustc artifacts, but in theory the loss isn't too bad!</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L292" href="#L292">292</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L293" href="#L293">293</a><span class="code-text">## Build tools</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L294" href="#L294">294</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L295" href="#L295">295</a><span class="code-text">We've actually got quite a few tools that we use in the compiler's build system</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L296" href="#L296">296</a><span class="code-text">and for testing. To organize these, each tool is a project in `src/tools` with a</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L297" href="#L297">297</a><span class="code-text">corresponding `Cargo.toml`. All tools are compiled with Cargo (currently having</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L298" href="#L298">298</a><span class="code-text">independent `Cargo.lock` files) and do not currently explicitly depend on the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L299" href="#L299">299</a><span class="code-text">compiler or standard library. Compiling each tool is sequenced after the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L300" href="#L300">300</a><span class="code-text">appropriate libstd/libtest/librustc compile above.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L301" href="#L301">301</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L302" href="#L302">302</a><span class="code-text">## Extending rustbuild</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L303" href="#L303">303</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L304" href="#L304">304</a><span class="code-text">So you'd like to add a feature to the rustbuild build system or just fix a bug.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L305" href="#L305">305</a><span class="code-text">Great! One of the major motivational factors for moving away from `make` is that</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L306" href="#L306">306</a><span class="code-text">Rust is in theory much easier to read, modify, and write. If you find anything</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L307" href="#L307">307</a><span class="code-text">excessively confusing, please open an issue on this and we'll try to get it</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L308" href="#L308">308</a><span class="code-text">documented or simplified pronto.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L309" href="#L309">309</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L310" href="#L310">310</a><span class="code-text">First up, you'll probably want to read over the documentation above as that'll</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L311" href="#L311">311</a><span class="code-text">give you a high level overview of what rustbuild is doing. You also probably</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L312" href="#L312">312</a><span class="code-text">want to play around a bit yourself by just getting it up and running before you</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L313" href="#L313">313</a><span class="code-text">dive too much into the actual build system itself.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L314" href="#L314">314</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L315" href="#L315">315</a><span class="code-text">After that, each module in rustbuild should have enough documentation to keep</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L316" href="#L316">316</a><span class="code-text">you up and running. Some general areas that you may be interested in modifying</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L317" href="#L317">317</a><span class="code-text">are:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L318" href="#L318">318</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L319" href="#L319">319</a><span class="code-text">* Adding a new build tool? Take a look at `bootstrap/tool.rs` for examples of</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L320" href="#L320">320</a><span class="code-text">  other tools.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L321" href="#L321">321</a><span class="code-text">* Adding a new compiler crate? Look no further! Adding crates can be done by</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L322" href="#L322">322</a><span class="code-text">  adding a new directory with `Cargo.toml` followed by configuring all</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L323" href="#L323">323</a><span class="code-text">  `Cargo.toml` files accordingly.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L324" href="#L324">324</a><span class="code-text">* Adding a new dependency from crates.io? This should just work inside the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L325" href="#L325">325</a><span class="code-text">  compiler artifacts stage (everything other than libtest and libstd).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L326" href="#L326">326</a><span class="code-text">* Adding a new configuration option? You'll want to modify `bootstrap/flags.rs`</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L327" href="#L327">327</a><span class="code-text">  for command line flags and then `bootstrap/config.rs` to copy the flags to the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L328" href="#L328">328</a><span class="code-text">  `Config` struct.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L329" href="#L329">329</a><span class="code-text">* Adding a sanity check? Take a look at `bootstrap/sanity.rs`.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L330" href="#L330">330</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L331" href="#L331">331</a><span class="code-text">If you have any questions feel free to reach out on `#infra` channel in the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L332" href="#L332">332</a><span class="code-text">[Rust Discord server][rust-discord] or ask on internals.rust-lang.org. When</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L333" href="#L333">333</a><span class="code-text">you encounter bugs, please file issues on the rust-lang/rust issue tracker.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L334" href="#L334">334</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L335" href="#L335">335</a><span class="code-text">[rust-discord]: https://discord.gg/rust-lang</span></span></pre></div></div>
</main>
</body>
</html>
