forked from symforce-org/symforce-org.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathdevelopment.html
297 lines (255 loc) · 22.4 KB
/
development.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Development Guide — symforce 0.5.0 documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
<link rel="stylesheet" type="text/css" href="_static/css/custom.css" />
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<link rel="shortcut icon" href="_static/favicon.ico"/>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="SymPy Tutorial" href="tutorials/sympy_tutorial.html" />
<link rel="prev" title="SymForce Home" href="index.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="development-guide">
<h1>Development Guide<a class="headerlink" href="#development-guide" title="Permalink to this headline">¶</a></h1>
<p>Guide for how to build, configure, and develop SymForce.</p>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#organization" id="id2">Organization</a></p></li>
<li><p><a class="reference internal" href="#build" id="id3">Build</a></p></li>
<li><p><a class="reference internal" href="#additional-useful-commands" id="id4">Additional useful commands</a></p></li>
<li><p><a class="reference internal" href="#documentation" id="id5">Documentation</a></p></li>
<li><p><a class="reference internal" href="#logging" id="id6">Logging</a></p></li>
<li><p><a class="reference internal" href="#testing-and-coverage" id="id7">Testing and Coverage</a></p></li>
<li><p><a class="reference internal" href="#formatting" id="id8">Formatting</a></p></li>
<li><p><a class="reference internal" href="#templates" id="id9">Templates</a></p></li>
<li><p><a class="reference internal" href="#symbolic-api" id="id10">Symbolic API</a></p></li>
<li><p><a class="reference internal" href="#building-wheels" id="id11">Building wheels</a></p></li>
</ul>
</div>
<section id="organization">
<h2><a class="toc-backref" href="#id2">Organization</a><a class="headerlink" href="#organization" title="Permalink to this headline">¶</a></h2>
<p>SymForce aims to follow Python <a class="reference external" href="https://docs.python-guide.org/writing/structure/">standards</a>. The core <code class="docutils literal notranslate"><span class="pre">symforce</span></code> package lives in the equivalently named subdirectory at the top level. Tests, documentation, etc live at the top level outside of the core package.
To import <code class="docutils literal notranslate"><span class="pre">symforce</span></code> add the top level to the Python path.</p>
<p>See the <a class="reference internal" href="index.html#api-reference"><span class="std std-ref">module reference</span></a> for the core package structure.</p>
</section>
<section id="build">
<h2><a class="toc-backref" href="#id3">Build</a><a class="headerlink" href="#build" title="Permalink to this headline">¶</a></h2>
<p>SymForce is primarily written in Python and C++, and is Python 3.8+ and C++14 compatible. The build
system is CMake for the C++ components, and optionally pip / setuptools on top for Python packaging.
See the Build section on the [Homepage](/index.html#build-from-source) for build instructions.</p>
</section>
<section id="additional-useful-commands">
<h2><a class="toc-backref" href="#id4">Additional useful commands</a><a class="headerlink" href="#additional-useful-commands" title="Permalink to this headline">¶</a></h2>
<p>SymForce also has a top level Makefile which is not used by the build, but provides some high
level commands for development:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 64%" />
<col style="width: 36%" />
</colgroup>
<tbody>
<tr class="row-odd"><td><p>Run Python tests</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Run tests which update (most) generated code</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test_update</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Run tests which update all generated code</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test_update_all</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Run tests and open coverage report</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">coverage_open</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Build docs</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">docs</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Build docs + open in browser</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">docs_open</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Run the code formatter (black, clang-format)</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">format</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Check types with mypy</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">check_types</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Check formatting and types</p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">lint</span></code></p></td>
</tr>
</tbody>
</table>
</section>
<section id="documentation">
<h2><a class="toc-backref" href="#id5">Documentation</a><a class="headerlink" href="#documentation" title="Permalink to this headline">¶</a></h2>
<p>This documentation is built with <a class="reference external" href="https://www.sphinx-doc.org/">Sphinx</a>, including automatic parsing of the code to generate a module reference using <a class="reference external" href="https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html">sphinx-apidoc</a>. The code uses <a class="reference external" href="https://www.sphinx-doc.org/en/1.6/ext/example_google.html">Google Style</a> docstrings to annotate all modules, classes, and functions. Docs pages are <code class="docutils literal notranslate"><span class="pre">.rst</span></code> files in <code class="docutils literal notranslate"><span class="pre">docs</span></code>, and the Sphinx config file is <code class="docutils literal notranslate"><span class="pre">docs/conf.py</span></code>.</p>
<p>There are sample <a class="reference external" href="https://jupyter.org/">Jupyter</a> notebooks in <code class="docutils literal notranslate"><span class="pre">notebooks</span></code>, some of which are built into these docs using <a class="reference external" href="https://nbsphinx.readthedocs.io/en/0.5.1/">nbsphinx</a>, such as the <a class="reference external" href="notebooks/tutorial.html">tutorial</a>. <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">notebook</span></code> starts a Jupyter server to modify and run them.</p>
</section>
<section id="logging">
<h2><a class="toc-backref" href="#id6">Logging</a><a class="headerlink" href="#logging" title="Permalink to this headline">¶</a></h2>
<p>SymForce uses the <a class="reference external" href="https://docs.python.org/2/library/logging.html">logging</a> module. You can import and use the logger like this:</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">symforce</span> <span class="kn">import</span> <span class="n">logger</span>
<span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'houston, we have a problem'</span><span class="p">)</span>
<span class="go">codegen_test.test_codegen_cpp():126 WARNING -- houston, we have a problem</span>
</pre></div>
</div>
<p>You can configure the log level using <a class="reference internal" href="api/symforce.html#symforce.set_log_level" title="symforce.set_log_level"><code class="xref py py-func docutils literal notranslate"><span class="pre">symforce.set_log_level()</span></code></a> or by setting the <code class="docutils literal notranslate"><span class="pre">SYMENGINE_LOGLEVEL</span></code> environment variable. The default is <code class="docutils literal notranslate"><span class="pre">logging.INFO</span></code>.</p>
</section>
<section id="testing-and-coverage">
<h2><a class="toc-backref" href="#id7">Testing and Coverage</a><a class="headerlink" href="#testing-and-coverage" title="Permalink to this headline">¶</a></h2>
<p>SymForce is heavily tested, targeting close to 100% code coverage.
Tests live in <code class="docutils literal notranslate"><span class="pre">test</span></code> and use <a class="reference external" href="https://docs.python.org/2/library/unittest.html">unittest</a>. Additionally, <a class="reference external" href="https://coverage.readthedocs.io/en/coverage-5.0.4/">coverage.py</a> is used to run tests while measuring code coverage. The generated coverage report also provides a great view into what methods need to be tested and what code is potentially unused.</p>
<div class="line-block">
<div class="line">Run all tests: <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test</span></code></div>
<div class="line">Run all tests and open the coverage report: <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">coverage_open</span></code></div>
<div class="line">Run a specific test: <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">test/symforce_codegen_test.py</span></code></div>
<div class="line">Run with debug level output: <code class="docutils literal notranslate"><span class="pre">SYMFORCE_LOGLEVEL=DEBUG</span> <span class="pre">python</span> <span class="pre">test/symforce_codegen_test.py</span></code></div>
</div>
<p>When debugging a specific test, the use of <a class="reference external" href="https://pypi.org/project/ipdb/">ipdb</a> is highly recommended, as is reproducing the most minimal example of the issue in a notebook.</p>
</section>
<section id="formatting">
<h2><a class="toc-backref" href="#id8">Formatting</a><a class="headerlink" href="#formatting" title="Permalink to this headline">¶</a></h2>
<p>Symforce uses the <a class="reference external" href="https://github.com/psf/black">Black</a> formatter for Python code. To quote the authors:</p>
<blockquote>
<div><p><cite>Black is the uncompromising Python code formatter. By using it, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from nagging about formatting. You will save time and mental energy for more important matters.</cite></p>
</div></blockquote>
<p>Running <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">format</span></code> will format the entire codebase. It’s recommended to develop with <a class="reference external" href="https://code.visualstudio.com/">VSCode</a> and integrate black.</p>
</section>
<section id="templates">
<h2><a class="toc-backref" href="#id9">Templates</a><a class="headerlink" href="#templates" title="Permalink to this headline">¶</a></h2>
<p>Much of the core functionality of SymForce is in generating code using the <a class="reference external" href="https://jinja.palletsprojects.com/en/2.11.x/">Jinja</a> template language. It’s relatively simple and easy to use - you pass it a template file in any language and a python dictionary of data, and it spits out the rendered code.</p>
<p>For example template files, see <code class="docutils literal notranslate"><span class="pre">symforce/codegen/backends/cpp/templates</span></code>.</p>
</section>
<section id="symbolic-api">
<h2><a class="toc-backref" href="#id10">Symbolic API</a><a class="headerlink" href="#symbolic-api" title="Permalink to this headline">¶</a></h2>
<p>SymForce uses the <a class="reference external" href="https://www.sympy.org/en/index.html">SymPy</a> API, but supports two implementations of it. The SymPy implementation is pure Python, whereas the <a class="reference external" href="https://github.com/symengine/symengine">SymEngine</a> implementation is wrapped C++. It can be 100-200 times faster for many operations, but is less fully featured and requires a C++ build.</p>
<p>To set the symbolic API, you can either use <a class="reference internal" href="api/symforce.html#symforce.set_symbolic_api" title="symforce.set_symbolic_api"><code class="xref py py-func docutils literal notranslate"><span class="pre">symforce.set_symbolic_api()</span></code></a> before any other imports, or use the <code class="docutils literal notranslate"><span class="pre">SYMFORCE_SYMBOLIC_API</span></code> environment variable with the options <code class="docutils literal notranslate"><span class="pre">sympy</span></code> or <code class="docutils literal notranslate"><span class="pre">symengine</span></code>. By default SymEngine will be used if found, otherwise SymPy.</p>
</section>
<section id="building-wheels">
<h2><a class="toc-backref" href="#id11">Building wheels</a><a class="headerlink" href="#building-wheels" title="Permalink to this headline">¶</a></h2>
<p>You should be able to build Python wheels of symforce the standard ways. We recommend using
<code class="docutils literal notranslate"><span class="pre">build</span></code>, i.e. running <code class="docutils literal notranslate"><span class="pre">python3</span> <span class="pre">-m</span> <span class="pre">build</span> <span class="pre">--wheel</span></code> from the <code class="docutils literal notranslate"><span class="pre">symforce</span></code> directory. By default,
this will build a wheel that includes local dependencies on the <code class="docutils literal notranslate"><span class="pre">skymarshal</span></code> and <code class="docutils literal notranslate"><span class="pre">symforce-sym</span></code>
packages (which are separate Python packages from <code class="docutils literal notranslate"><span class="pre">symforce</span></code> itself). For distribution, you’ll
typically want to set the environment variable <code class="docutils literal notranslate"><span class="pre">SYMFORCE_REWRITE_LOCAL_DEPENDENCIES=True</span></code> when
building, and also run <code class="docutils literal notranslate"><span class="pre">python3</span> <span class="pre">-m</span> <span class="pre">build</span> <span class="pre">--wheel</span> <span class="pre">third_party/skymarshal</span></code> and
<code class="docutils literal notranslate"><span class="pre">python3</span> <span class="pre">-m</span> <span class="pre">build</span> <span class="pre">--wheel</span> <span class="pre">gen/python</span></code> to build wheels for those packages separately.</p>
<p>For SymForce releases, all of this is handled by the <code class="docutils literal notranslate"><span class="pre">build_wheels</span></code> GitHub Actions workflow. This
workflow is currently run manually on a commit, and produces a <code class="docutils literal notranslate"><span class="pre">symforce-wheels.zip</span></code> artifact with
wheels (and sdists) for distribution (e.g. on PyPI). It doesn’t upload them to PyPI - to do that
(after verifying that the built wheels work as expected) you should download and unzip the archive,
and upload to PyPI with <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">twine</span> <span class="pre">upload</span> <span class="pre">[--repository</span> <span class="pre">testpypi]</span> <span class="pre">--verbose</span> <span class="pre">*</span></code>.</p>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">symforce</a></h1>
<p class="blurb">Fast symbolic computation, code generation, and nonlinear optimization for robotics</p>
<p>
<iframe src="https://ghbtns.com/github-btn.html?user=symforce-org&repo=symforce&type=star&count=true&size=large&v=2"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<h3>Navigation</h3>
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Development Guide</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#organization">Organization</a></li>
<li class="toctree-l2"><a class="reference internal" href="#build">Build</a></li>
<li class="toctree-l2"><a class="reference internal" href="#additional-useful-commands">Additional useful commands</a></li>
<li class="toctree-l2"><a class="reference internal" href="#documentation">Documentation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#logging">Logging</a></li>
<li class="toctree-l2"><a class="reference internal" href="#testing-and-coverage">Testing and Coverage</a></li>
<li class="toctree-l2"><a class="reference internal" href="#formatting">Formatting</a></li>
<li class="toctree-l2"><a class="reference internal" href="#templates">Templates</a></li>
<li class="toctree-l2"><a class="reference internal" href="#symbolic-api">Symbolic API</a></li>
<li class="toctree-l2"><a class="reference internal" href="#building-wheels">Building wheels</a></li>
</ul>
</li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Tutorials</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/sympy_tutorial.html">SymPy Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/geometry_tutorial.html">Geometry Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/ops_tutorial.html">Ops Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/cameras_tutorial.html">Cameras Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/values_tutorial.html">Values Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/codegen_tutorial.html">Codegen Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/optimization_tutorial.html">Optimization Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorials/epsilon_tutorial.html">Epsilon Tutorial</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Examples</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="examples/bundle_adjustment/README.html">Bundle Adjustment</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples/bundle_adjustment_fixed_size/README.html">Fixed Size Bundle Adjustment</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples/bundle_adjustment_in_the_large/README.html">Bundle-Adjustment-in-the-Large</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples/custom_factor_generation/README.html">Custom Factor Generation</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples/robot_2d_localization/README.html">Robot 2D Localization</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples/robot_3d_localization/README.html">Robot 3D Localization</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">symforce Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api/symforce.html">symforce package</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">sym Python Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api-gen-py/sym.html">sym package</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">sym C++ Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api-gen-cpp/classlist.html">Class list</a></li>
<li class="toctree-l1"><a class="reference internal" href="api-gen-cpp/filelist.html">File list</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">opt C++ Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api-cpp/classlist.html">Class list</a></li>
<li class="toctree-l1"><a class="reference internal" href="api-cpp/filelist.html">File list</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="index.html" title="previous chapter">SymForce Home</a></li>
<li>Next: <a href="tutorials/sympy_tutorial.html" title="next chapter">SymPy Tutorial</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2022, Skydio, Inc.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 4.5.0</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="_sources/development.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>