<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>src/tools/rustfmt/Design.md - fork/rust @ master</title>
<meta name="description" content="src/tools/rustfmt/Design.md in fork/rust at master.">
<meta property="og:title" content="src/tools/rustfmt/Design.md - fork/rust @ master">
<meta property="og:description" content="src/tools/rustfmt/Design.md in fork/rust at master.">
<meta property="og:type" content="website">
<meta property="og:site_name" content="git.julia.blue">
<link rel="stylesheet" href="/css/style.css?v=62">
<script type="module" src="/js/main.js?v=62"></script>
</head>
<body>
<main data-ref="master" data-tip="3bcad65fbf67b6edf87c949531fbc13435882e6b">
<noscript>
  <div class="callout">
    <p>
      hey! this is a static git repo viewer. commit diffs, file history,
      older commits and other revisions won't load without javascript.
    </p>
    <p>
      you are still able to view source files on branch/tag tips,
      but the experience is degraded.
    </p>
  </div>
</noscript>
<header class="repo"><div>
<h1>fork/rust</h1>
</div>
<pre class="clone">git clone https://git.julia.blue/fork/rust</pre>
</header>
<p class="snapshot-commit"><span class="commit-author">bors</span><span class="commit-message" data-commit="3bcad65fbf67b6edf87c949531fbc13435882e6b">Auto merge of #103046 - JanBeh:PR_clarify_cmp_terminology, r=workingjubilee</span><span class="snapshot-sha sha" data-commit="3bcad65fbf67b6edf87c949531fbc13435882e6b">3bcad65fbf6</span><time datetime="2023-10-05T15:24:54+00:00" title="2023-10-05 15:24:54 +0000">2023-10-05</time></p>
<header class="topbar"><details class="switcher"><summary>master</summary><div class="menu"><strong>branches</strong><a class="current" href="/fork/rust/ref/master/">master</a><strong>tags</strong><a href="/fork/rust/ref/1.39.0/">1.39.0</a><a href="/fork/rust/ref/1.40.0/">1.40.0</a><a href="/fork/rust/ref/1.41.0/">1.41.0</a><a href="/fork/rust/ref/1.41.1/">1.41.1</a><a href="/fork/rust/ref/1.42.0/">1.42.0</a><a href="/fork/rust/ref/1.43.0/">1.43.0</a><a href="/fork/rust/ref/1.43.1/">1.43.1</a><a href="/fork/rust/ref/1.44.0/">1.44.0</a><a class="all" href="/fork/rust/refs">all refs →</a></div></details><nav class="crumbs"><a href="/fork/rust/">rust</a> / <a href="/fork/rust/ref/master/tree/src/">src</a> / <a href="/fork/rust/ref/master/tree/src/tools/">tools</a> / <a href="/fork/rust/ref/master/tree/src/tools/rustfmt/">rustfmt</a> / <span class="cur">Design.md</span></nav>
<span class="view-stats"><span>8.5 KiB</span><span>184 lines</span></span><span class="actions"><a class="raw" href="/fork/rust/raw/7a4dcf8773b61a85b7fc6fcc6dd08c9a91b55a00/src/tools/rustfmt/Design.md">raw</a></span></header>
<div class="markdown-view"><input type="radio" name="markdown-view" id="markdown-rendered" checked><label for="markdown-rendered">rendered</label><input type="radio" name="markdown-view" id="markdown-code"><label for="markdown-code">code</label><section class="readme rendered"><h1>Some thoughts on the design of rustfmt</h1>
<h2>Use cases</h2>
<p>A formatting tool can be used in different ways and the different use cases can
affect the design of the tool. The use cases I'm particularly concerned with are:</p>
<ul>
<li>running on a whole repo before check-in
<ul>
<li>in particular, to replace the <code>make tidy</code> pass on the Rust distro</li>
</ul>
</li>
<li>running on code from another project that you are adding to your own</li>
<li>using for mass changes in code style over a project</li>
</ul>
<p>Some valid use cases for a formatting tool which I am explicitly not trying to
address (although it would be nice, if possible):</p>
<ul>
<li>running 'as you type' in an IDE</li>
<li>running on arbitrary snippets of code</li>
<li>running on Rust-like code, specifically code which doesn't parse</li>
<li>use as a pretty printer inside the compiler</li>
<li>refactoring</li>
<li>formatting totally unformatted source code</li>
</ul>
<h2>Scope and vision</h2>
<p>I do not subscribe to the notion that a formatting tool should only change
whitespace. I believe that we should semantics preserving, but not necessarily
syntax preserving, i.e., we can change the AST of a program.</p>
<p>I.e., we might change glob imports to list or single imports, re-order imports,
move bounds to where clauses, combine multiple impls into a single impl, etc.</p>
<p>However, we will not change the names of variables or make any changes which
<em>could</em> change the semantics. To be ever so slightly formal, we might imagine
a compilers high level intermediate representation, we should strive to only
make changes which do not change the HIR, even if they do change the AST.</p>
<p>I would like to be able to output refactoring scripts for making deeper changes
though. (E.g., renaming variables to satisfy our style guidelines).</p>
<p>My long term goal is that all style lints can be moved from the compiler to
rustfmt and, as well as warning, can either fix problems or emit refactoring
scripts to do so.</p>
<h3>Configurability</h3>
<p>I believe reformatting should be configurable to some extent. We should read in
options from a configuration file and reformat accordingly. We should supply at
least a config file which matches the Rust style guidelines.</p>
<p>There should be multiple modes for running the tool. As well as simply replacing
each file, we should be able to show the user a list of the changes we would
make, or show a list of violations without corrections (the difference being
that there are multiple ways to satisfy a given set of style guidelines, and we
should distinguish violations from deviations from our own model).</p>
<h2>Implementation philosophy</h2>
<p>Some details of the philosophy behind the implementation.</p>
<h3>Operate on the AST</h3>
<p>A reformatting tool can be based on either the AST or a token stream (in Rust
this is actually a stream of token trees, but it's not a fundamental difference).
There are pros and cons to the two approaches. I have chosen to use the AST
approach. The primary reasons are that it allows us to do more sophisticated
manipulations, rather than just change whitespace, and it gives us more context
when making those changes.</p>
<p>The advantage of the tokens approach is that you can operate on non-parsable
code. I don't care too much about that, it would be nice, but I think being able
to perform sophisticated transformations is more important. In the future, I hope to
(optionally) be able to use type information for informing reformatting too. One
specific case of unparsable code is macros. Using tokens is certainly easier
here, but I believe it is perfectly solvable with the AST approach. At the limit,
we can operate on just tokens in the macro case.</p>
<p>I believe that there is not in fact that much difference between the two
approaches. Due to imperfect span information, under the AST approach, we
sometimes are reduced to examining tokens or do some re-lexing of our own. Under
the tokens approach, you need to implement your own (much simpler) parser. I
believe that as the tool gets more sophisticated, you end up doing more at the
token-level, or having an increasingly sophisticated parser, until at the limit
you have the same tool.</p>
<p>However, I believe starting from the AST gets you more quickly to a usable and
useful tool.</p>
<h3>Heuristic rather than algorithmic</h3>
<p>Many formatting tools use a very general algorithmic or even algebraic tool for
pretty printing. This results in very elegant code, but I believe does not give
the best results. I prefer a more ad hoc approach where each expression/item is
formatted using custom rules. We hopefully don't end up with too much code due
to good old fashioned abstraction and code sharing. This will give a bigger code
base, but hopefully a better result.</p>
<p>It also means that there will be some cases we can't format and we have to give
up. I think that is OK. Hopefully, they are rare enough that manually fixing them
is not painful. Better to have a tool that gives great code in 99% of cases and
fails in 1% than a tool which gives 50% great code and 50% ugly code, but never
fails.</p>
<h3>Incremental development</h3>
<p>I want rustfmt to be useful as soon as possible and to always be useful. I
specifically don't want to have to wait for a feature (or worse, the whole tool)
to be perfect before it is useful. The main ways this is achieved is to output
the source code where we can't yet reformat, be able to turn off new features
until they are ready, and the 'do no harm' principle (see next section).</p>
<h3>First, do no harm</h3>
<p>Until rustfmt is perfect, there will always be a trade-off between doing more and
doing existing things well. I want to err on the side of the latter.
Specifically, rustfmt should never take OK code and make it look worse. If we
can't make it better, we should leave it as is. That might mean being less
aggressive than we like or using configurability.</p>
<h3>Use the source code as guidance</h3>
<p>There are often multiple ways to format code and satisfy standards. Where this
is the case, we should use the source code as a hint for reformatting.
Furthermore, where the code has been formatted in a particular way that satisfies
the coding standard, it should not be changed (this is sometimes not possible or
not worthwhile due to uniformity being desirable, but it is a useful goal).</p>
<h3>Architecture details</h3>
<p>We use the AST from <a href="https://crates.io/crates/syntex_syntax">syntex_syntax</a>, an export of rustc's libsyntax. We use
syntex_syntax's visit module to walk the AST to find starting points for
reformatting. Eventually, we should reformat everything and we shouldn't need
the visit module. We keep track of the last formatted position in the code, and
when we reformat the next piece of code we make sure to output the span for all
the code in between (handled by missed_spans.rs).</p>
<p>We read in formatting configuration from a <code>rustfmt.toml</code> file if there is one.
The options and their defaults are defined in <code>config.rs</code>. A <code>Config</code> object is
passed throughout the formatting code, and each formatting routine looks there
for its configuration.</p>
<p>Our visitor keeps track of the desired current indent due to blocks (
<code>block_indent</code>). Each <code>visit_*</code> method reformats code according to this indent,
<code>config.comment_width()</code> and <code>config.max_width()</code>. Most reformatting that is done
in the <code>visit_*</code> methods is a bit hacky and is meant to be temporary until it can
be done properly.</p>
<p>There are a bunch of methods called <code>rewrite_*</code>. They do the bulk of the
reformatting. These take the AST node to be reformatted (this may not literally
be an AST node from syntex_syntax: there might be multiple parameters
describing a logical node), the current indent, and the current width budget.
They return a <code>String</code> (or sometimes an <code>Option&lt;String&gt;</code>) which formats the
code in the box given by the indent and width budget. If the method fails, it
returns <code>None</code> and the calling method then has to fallback in some way to give
the callee more space.</p>
<p>So, in summary, to format a node, we calculate the width budget and then walk down
the tree from the node. At a leaf, we generate an actual string and then unwind,
combining these strings as we go back up the tree.</p>
<p>For example, consider a method definition:</p>
<pre>    fn foo(a: A, b: B) {
        ...
    }
</pre>
<p>We start at indent 4, the rewrite function for the whole function knows it must
write <code>fn foo(</code> before the arguments and <code>) {</code> after them, assuming the max width
is 100, it thus asks the rewrite argument list function to rewrite with an indent
of 11 and in a width of 86. Assuming that is possible (obviously in this case),
it returns a string for the arguments and it can make a string for the function
header. If the arguments couldn't be fitted in that space, we might try to
fallback to a hanging indent, so we try again with indent 8 and width 89.</p>
</section><div class="code-panel"><pre class="code src" data-hl="src/tools/rustfmt/Design.md"><span class="code-line"><a class="ln" id="L1" href="#L1">1</a><span class="code-text"># Some thoughts on the design of rustfmt</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L2" href="#L2">2</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L3" href="#L3">3</a><span class="code-text">## Use cases</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L4" href="#L4">4</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L5" href="#L5">5</a><span class="code-text">A formatting tool can be used in different ways and the different use cases can</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L6" href="#L6">6</a><span class="code-text">affect the design of the tool. The use cases I'm particularly concerned with are:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L7" href="#L7">7</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L8" href="#L8">8</a><span class="code-text">* running on a whole repo before check-in</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L9" href="#L9">9</a><span class="code-text">  - in particular, to replace the `make tidy` pass on the Rust distro</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L10" href="#L10">10</a><span class="code-text">* running on code from another project that you are adding to your own</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L11" href="#L11">11</a><span class="code-text">* using for mass changes in code style over a project</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L12" href="#L12">12</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L13" href="#L13">13</a><span class="code-text">Some valid use cases for a formatting tool which I am explicitly not trying to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L14" href="#L14">14</a><span class="code-text">address (although it would be nice, if possible):</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L15" href="#L15">15</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L16" href="#L16">16</a><span class="code-text">* running 'as you type' in an IDE</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L17" href="#L17">17</a><span class="code-text">* running on arbitrary snippets of code</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L18" href="#L18">18</a><span class="code-text">* running on Rust-like code, specifically code which doesn't parse</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L19" href="#L19">19</a><span class="code-text">* use as a pretty printer inside the compiler</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L20" href="#L20">20</a><span class="code-text">* refactoring</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L21" href="#L21">21</a><span class="code-text">* formatting totally unformatted source code</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L22" href="#L22">22</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L23" href="#L23">23</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L24" href="#L24">24</a><span class="code-text">## Scope and vision</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L25" href="#L25">25</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L26" href="#L26">26</a><span class="code-text">I do not subscribe to the notion that a formatting tool should only change</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L27" href="#L27">27</a><span class="code-text">whitespace. I believe that we should semantics preserving, but not necessarily</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L28" href="#L28">28</a><span class="code-text">syntax preserving, i.e., we can change the AST of a program.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L29" href="#L29">29</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L30" href="#L30">30</a><span class="code-text">I.e., we might change glob imports to list or single imports, re-order imports,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L31" href="#L31">31</a><span class="code-text">move bounds to where clauses, combine multiple impls into a single impl, etc.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L32" href="#L32">32</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L33" href="#L33">33</a><span class="code-text">However, we will not change the names of variables or make any changes which</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L34" href="#L34">34</a><span class="code-text">*could* change the semantics. To be ever so slightly formal, we might imagine</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L35" href="#L35">35</a><span class="code-text">a compilers high level intermediate representation, we should strive to only</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L36" href="#L36">36</a><span class="code-text">make changes which do not change the HIR, even if they do change the AST.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L37" href="#L37">37</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L38" href="#L38">38</a><span class="code-text">I would like to be able to output refactoring scripts for making deeper changes</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L39" href="#L39">39</a><span class="code-text">though. (E.g., renaming variables to satisfy our style guidelines).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L40" href="#L40">40</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L41" href="#L41">41</a><span class="code-text">My long term goal is that all style lints can be moved from the compiler to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L42" href="#L42">42</a><span class="code-text">rustfmt and, as well as warning, can either fix problems or emit refactoring</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L43" href="#L43">43</a><span class="code-text">scripts to do so.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L44" href="#L44">44</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L45" href="#L45">45</a><span class="code-text">### Configurability</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L46" href="#L46">46</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L47" href="#L47">47</a><span class="code-text">I believe reformatting should be configurable to some extent. We should read in</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L48" href="#L48">48</a><span class="code-text">options from a configuration file and reformat accordingly. We should supply at</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L49" href="#L49">49</a><span class="code-text">least a config file which matches the Rust style guidelines.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L50" href="#L50">50</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L51" href="#L51">51</a><span class="code-text">There should be multiple modes for running the tool. As well as simply replacing</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L52" href="#L52">52</a><span class="code-text">each file, we should be able to show the user a list of the changes we would</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L53" href="#L53">53</a><span class="code-text">make, or show a list of violations without corrections (the difference being</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L54" href="#L54">54</a><span class="code-text">that there are multiple ways to satisfy a given set of style guidelines, and we</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L55" href="#L55">55</a><span class="code-text">should distinguish violations from deviations from our own model).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L56" href="#L56">56</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L57" href="#L57">57</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L58" href="#L58">58</a><span class="code-text">## Implementation philosophy</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L59" href="#L59">59</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L60" href="#L60">60</a><span class="code-text">Some details of the philosophy behind the implementation.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L61" href="#L61">61</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L62" href="#L62">62</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L63" href="#L63">63</a><span class="code-text">### Operate on the AST</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L64" href="#L64">64</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L65" href="#L65">65</a><span class="code-text">A reformatting tool can be based on either the AST or a token stream (in Rust</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L66" href="#L66">66</a><span class="code-text">this is actually a stream of token trees, but it's not a fundamental difference).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L67" href="#L67">67</a><span class="code-text">There are pros and cons to the two approaches. I have chosen to use the AST</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L68" href="#L68">68</a><span class="code-text">approach. The primary reasons are that it allows us to do more sophisticated</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L69" href="#L69">69</a><span class="code-text">manipulations, rather than just change whitespace, and it gives us more context</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L70" href="#L70">70</a><span class="code-text">when making those changes.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L71" href="#L71">71</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L72" href="#L72">72</a><span class="code-text">The advantage of the tokens approach is that you can operate on non-parsable</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L73" href="#L73">73</a><span class="code-text">code. I don't care too much about that, it would be nice, but I think being able</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L74" href="#L74">74</a><span class="code-text">to perform sophisticated transformations is more important. In the future, I hope to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L75" href="#L75">75</a><span class="code-text">(optionally) be able to use type information for informing reformatting too. One</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L76" href="#L76">76</a><span class="code-text">specific case of unparsable code is macros. Using tokens is certainly easier</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L77" href="#L77">77</a><span class="code-text">here, but I believe it is perfectly solvable with the AST approach. At the limit,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L78" href="#L78">78</a><span class="code-text">we can operate on just tokens in the macro case.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L79" href="#L79">79</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L80" href="#L80">80</a><span class="code-text">I believe that there is not in fact that much difference between the two</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L81" href="#L81">81</a><span class="code-text">approaches. Due to imperfect span information, under the AST approach, we</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L82" href="#L82">82</a><span class="code-text">sometimes are reduced to examining tokens or do some re-lexing of our own. Under</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L83" href="#L83">83</a><span class="code-text">the tokens approach, you need to implement your own (much simpler) parser. I</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L84" href="#L84">84</a><span class="code-text">believe that as the tool gets more sophisticated, you end up doing more at the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L85" href="#L85">85</a><span class="code-text">token-level, or having an increasingly sophisticated parser, until at the limit</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L86" href="#L86">86</a><span class="code-text">you have the same tool.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L87" href="#L87">87</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L88" href="#L88">88</a><span class="code-text">However, I believe starting from the AST gets you more quickly to a usable and</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L89" href="#L89">89</a><span class="code-text">useful tool.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L90" href="#L90">90</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L91" href="#L91">91</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L92" href="#L92">92</a><span class="code-text">### Heuristic rather than algorithmic</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L93" href="#L93">93</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L94" href="#L94">94</a><span class="code-text">Many formatting tools use a very general algorithmic or even algebraic tool for</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L95" href="#L95">95</a><span class="code-text">pretty printing. This results in very elegant code, but I believe does not give</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L96" href="#L96">96</a><span class="code-text">the best results. I prefer a more ad hoc approach where each expression/item is</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L97" href="#L97">97</a><span class="code-text">formatted using custom rules. We hopefully don't end up with too much code due</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L98" href="#L98">98</a><span class="code-text">to good old fashioned abstraction and code sharing. This will give a bigger code</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L99" href="#L99">99</a><span class="code-text">base, but hopefully a better result.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L100" href="#L100">100</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L101" href="#L101">101</a><span class="code-text">It also means that there will be some cases we can't format and we have to give</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L102" href="#L102">102</a><span class="code-text">up. I think that is OK. Hopefully, they are rare enough that manually fixing them</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L103" href="#L103">103</a><span class="code-text">is not painful. Better to have a tool that gives great code in 99% of cases and</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L104" href="#L104">104</a><span class="code-text">fails in 1% than a tool which gives 50% great code and 50% ugly code, but never</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L105" href="#L105">105</a><span class="code-text">fails.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L106" href="#L106">106</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L107" href="#L107">107</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L108" href="#L108">108</a><span class="code-text">### Incremental development</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L109" href="#L109">109</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L110" href="#L110">110</a><span class="code-text">I want rustfmt to be useful as soon as possible and to always be useful. I</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L111" href="#L111">111</a><span class="code-text">specifically don't want to have to wait for a feature (or worse, the whole tool)</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L112" href="#L112">112</a><span class="code-text">to be perfect before it is useful. The main ways this is achieved is to output</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L113" href="#L113">113</a><span class="code-text">the source code where we can't yet reformat, be able to turn off new features</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L114" href="#L114">114</a><span class="code-text">until they are ready, and the 'do no harm' principle (see next section).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L115" href="#L115">115</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L116" href="#L116">116</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L117" href="#L117">117</a><span class="code-text">### First, do no harm</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L118" href="#L118">118</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L119" href="#L119">119</a><span class="code-text">Until rustfmt is perfect, there will always be a trade-off between doing more and</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L120" href="#L120">120</a><span class="code-text">doing existing things well. I want to err on the side of the latter.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L121" href="#L121">121</a><span class="code-text">Specifically, rustfmt should never take OK code and make it look worse. If we</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L122" href="#L122">122</a><span class="code-text">can't make it better, we should leave it as is. That might mean being less</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L123" href="#L123">123</a><span class="code-text">aggressive than we like or using configurability.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L124" href="#L124">124</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L125" href="#L125">125</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L126" href="#L126">126</a><span class="code-text">### Use the source code as guidance</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L127" href="#L127">127</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L128" href="#L128">128</a><span class="code-text">There are often multiple ways to format code and satisfy standards. Where this</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L129" href="#L129">129</a><span class="code-text">is the case, we should use the source code as a hint for reformatting.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L130" href="#L130">130</a><span class="code-text">Furthermore, where the code has been formatted in a particular way that satisfies</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L131" href="#L131">131</a><span class="code-text">the coding standard, it should not be changed (this is sometimes not possible or</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L132" href="#L132">132</a><span class="code-text">not worthwhile due to uniformity being desirable, but it is a useful goal).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L133" href="#L133">133</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L134" href="#L134">134</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L135" href="#L135">135</a><span class="code-text">### Architecture details</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L136" href="#L136">136</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L137" href="#L137">137</a><span class="code-text">We use the AST from [syntex_syntax], an export of rustc's libsyntax. We use</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L138" href="#L138">138</a><span class="code-text">syntex_syntax's visit module to walk the AST to find starting points for</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L139" href="#L139">139</a><span class="code-text">reformatting. Eventually, we should reformat everything and we shouldn't need</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L140" href="#L140">140</a><span class="code-text">the visit module. We keep track of the last formatted position in the code, and</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L141" href="#L141">141</a><span class="code-text">when we reformat the next piece of code we make sure to output the span for all</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L142" href="#L142">142</a><span class="code-text">the code in between (handled by missed_spans.rs).</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L143" href="#L143">143</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L144" href="#L144">144</a><span class="code-text">[syntex_syntax]: https://crates.io/crates/syntex_syntax</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L145" href="#L145">145</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L146" href="#L146">146</a><span class="code-text">We read in formatting configuration from a `rustfmt.toml` file if there is one.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L147" href="#L147">147</a><span class="code-text">The options and their defaults are defined in `config.rs`. A `Config` object is</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L148" href="#L148">148</a><span class="code-text">passed throughout the formatting code, and each formatting routine looks there</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L149" href="#L149">149</a><span class="code-text">for its configuration.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L150" href="#L150">150</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L151" href="#L151">151</a><span class="code-text">Our visitor keeps track of the desired current indent due to blocks (</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L152" href="#L152">152</a><span class="code-text">`block_indent`). Each `visit_*` method reformats code according to this indent,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L153" href="#L153">153</a><span class="code-text">`config.comment_width()` and `config.max_width()`. Most reformatting that is done</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L154" href="#L154">154</a><span class="code-text">in the `visit_*` methods is a bit hacky and is meant to be temporary until it can</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L155" href="#L155">155</a><span class="code-text">be done properly.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L156" href="#L156">156</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L157" href="#L157">157</a><span class="code-text">There are a bunch of methods called `rewrite_*`. They do the bulk of the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L158" href="#L158">158</a><span class="code-text">reformatting. These take the AST node to be reformatted (this may not literally</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L159" href="#L159">159</a><span class="code-text">be an AST node from syntex_syntax: there might be multiple parameters</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L160" href="#L160">160</a><span class="code-text">describing a logical node), the current indent, and the current width budget.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L161" href="#L161">161</a><span class="code-text">They return a `String` (or sometimes an `Option&lt;String&gt;`) which formats the</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L162" href="#L162">162</a><span class="code-text">code in the box given by the indent and width budget. If the method fails, it</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L163" href="#L163">163</a><span class="code-text">returns `None` and the calling method then has to fallback in some way to give</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L164" href="#L164">164</a><span class="code-text">the callee more space.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L165" href="#L165">165</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L166" href="#L166">166</a><span class="code-text">So, in summary, to format a node, we calculate the width budget and then walk down</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L167" href="#L167">167</a><span class="code-text">the tree from the node. At a leaf, we generate an actual string and then unwind,</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L168" href="#L168">168</a><span class="code-text">combining these strings as we go back up the tree.</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L169" href="#L169">169</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L170" href="#L170">170</a><span class="code-text">For example, consider a method definition:</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L171" href="#L171">171</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L172" href="#L172">172</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L173" href="#L173">173</a><span class="code-text">    fn foo(a: A, b: B) {</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L174" href="#L174">174</a><span class="code-text">        ...</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L175" href="#L175">175</a><span class="code-text">    }</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L176" href="#L176">176</a><span class="code-text">```</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L177" href="#L177">177</a><span class="code-text"></span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L178" href="#L178">178</a><span class="code-text">We start at indent 4, the rewrite function for the whole function knows it must</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L179" href="#L179">179</a><span class="code-text">write `fn foo(` before the arguments and `) {` after them, assuming the max width</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L180" href="#L180">180</a><span class="code-text">is 100, it thus asks the rewrite argument list function to rewrite with an indent</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L181" href="#L181">181</a><span class="code-text">of 11 and in a width of 86. Assuming that is possible (obviously in this case),</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L182" href="#L182">182</a><span class="code-text">it returns a string for the arguments and it can make a string for the function</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L183" href="#L183">183</a><span class="code-text">header. If the arguments couldn't be fitted in that space, we might try to</span><span class="code-break">
</span></span><span class="code-line"><a class="ln" id="L184" href="#L184">184</a><span class="code-text">fallback to a hanging indent, so we try again with indent 8 and width 89.</span></span></pre></div></div>
</main>
</body>
</html>
