forked from python/peps
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpep-0441.txt
361 lines (276 loc) · 14.3 KB
/
pep-0441.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
PEP: 441
Title: Improving Python ZIP Application Support
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <[email protected]>,
Paul Moore <[email protected]>
Discussions-To: https://mail.python.org/pipermail/python-dev/2015-February/138277.html
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 30-Mar-2013
Post-History: 30-Mar-2013, 01-Apr-2013, 16-Feb-2015
Resolution: https://mail.python.org/pipermail/python-dev/2015-February/138578.html
Improving Python ZIP Application Support
========================================
Python has had the ability to execute directories or ZIP-format
archives as scripts since version 2.6 [1]_. When invoked with a zip
file or directory as its first argument the interpreter adds that
directory to sys.path and executes the ``__main__`` module. These
archives provide a great way to publish software that needs to be
distributed as a single file script but is complex enough to need to
be written as a collection of modules.
This feature is not as popular as it should be mainly because it was
not promoted as part of Python 2.6 [2]_, so that it is relatively
unknown, but also because the Windows installer does not register a
file extension (other than ``.py``) for this format of file, to associate
with the launcher.
This PEP proposes to fix these problems by re-publicising the feature,
defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications"
and "Windowed Python ZIP Applications", and providing some simple
tooling to manage the format.
A New Python ZIP Application Extension
======================================
The terminology "Python Zip Application" will be the formal term used
for a zip-format archive that contains Python code in a form that can
be directly executed by Python (specifically, it must have a
``__main__.py`` file in the root directory of the archive). The
extension ``.pyz`` will be formally associated with such files.
The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python
Zip Applications" with the platform launcher so they can be executed.
A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a
windowed application, indicating whether the console should appear
when running the app.
On Unix, it would be ideal if the ``.pyz`` extension and the name
"Python Zip Application" were registered (in the mime types database?).
However, such an association is out of scope for this PEP.
Python Zip applications can be prefixed with a ``#!`` line
pointing to the correct Python interpreter and an optional
explanation::
#!/usr/bin/env python3
# Python application packed with zipapp module
(binary contents of archive)
On Unix, this allows the OS to run the file with the correct
interpreter, via the standard "shebang" support. On Windows, the
Python launcher implements shebang support.
However, it is always possible to execute a ``.pyz`` application by
supplying the filename to the Python interpreter directly.
As background, ZIP archives are defined with a footer containing
relative offsets from the end of the file. They remain valid when
concatenated to the end of any other file. This feature is completely
standard and is how self-extracting ZIP archives and the bdist_wininst
installer format work.
Minimal Tooling: The zipapp Module
==================================
This PEP also proposes including a module for working with these
archives. The module will contain functions for working with Python
zip application archives, and a command line interface (via ``python
-m zipapp``) for their creation and manipulation.
More complete tools for managing Python Zip Applications are
encouraged as 3rd party applications on PyPI. Currently, pyzzer [5]_
and pex [6]_ are two such tools.
Module Interface
----------------
The zipapp module will provide the following functions:
``create_archive(source, target=None, interpreter=None, main=None)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Create an application archive from *source*. The source can be any
of the following:
* The name of a directory, in which case a new application archive
will be created from the content of that directory.
* The name of an existing application archive file, in which case the
file is copied to the target. The file name should include the
``.pyz`` or ``.pyzw`` extension, if required.
* A file object open for reading in bytes mode. The content of the
file should be an application archive, and the file object is
assumed to be positioned at the start of the archive.
The *target* argument determines where the resulting archive will be
written:
* If it is the name of a file, the archive will be written to that
file.
* If it is an open file object, the archive will be written to that
file object, which must be open for writing in bytes mode.
* If the target is omitted (or None), the source must be a directory
and the target will be a file with the same name as the source, with
a ``.pyz`` extension added.
The *interpreter* argument specifies the name of the Python
interpreter with which the archive will be executed. It is written as
a "shebang" line at the start of the archive. On Unix, this will be
interpreted by the OS, and on Windows it will be handled by the Python
launcher. Omitting the *interpreter* results in no shebang line being
written. If an interpreter is specified, and the target is a
filename, the executable bit of the target file will be set.
The *main* argument specifies the name of a callable which will be
used as the main program for the archive. It can only be specified if
the source is a directory, and the source does not already contain a
``__main__.py`` file. The *main* argument should take the form
"pkg.module:callable" and the archive will be run by importing
"pkg.module" and executing the given callable with no arguments. It
is an error to omit *main* if the source is a directory and does not
contain a ``__main__.py`` file, as otherwise the resulting archive
would not be executable.
If a file object is specified for *source* or *target*, it is the
caller's responsibility to close it after calling create_archive.
When copying an existing archive, file objects supplied only need
``read`` and ``readline``, or ``write`` methods. When creating an
archive from a directory, if the target is a file object it will be
passed to the ``zipfile.ZipFile`` class, and must supply the methods
needed by that class.
``get_interpreter(archive)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Returns the interpreter specified in the shebang line of the
*archive*. If there is no shebang, the function returns ``None``.
The *archive* argument can be a filename or a file-like object open
for reading in bytes mode.
Command Line Usage
------------------
The zipapp module can be run with the python ``-m`` flag. The command
line interface is as follows::
python -m zipapp directory [options]
Create an archive from the given directory. An archive will
be created from the contents of that directory. The archive
will have the same name as the source directory with a .pyz
extension.
The following options can be specified:
-o archive / --output archive
The destination archive will have the specified name. The
given name will be used as written, so should include the
".pyz" or ".pyzw" extension.
-p interpreter / --python interpreter
The given interpreter will be written to the shebang line
of the archive. If this option is not given, the archive
will have no shebang line.
-m pkg.mod:fn / --main pkg.mod:fn
The source directory must not have a __main__.py file. The
archiver will write a __main__.py file into the target
which calls fn from the module pkg.mod.
The behaviour of the command line interface matches that of
``zipapp.create_archive()``.
In addition, it is possible to use the command line interface to work
with an existing archive::
python -m zipapp app.pyz --show
Displays the shebang line of an archive. Output is of the
form
Interpreter: /usr/bin/env
or
Interpreter: <none>
and is intended for diagnostic use, not for scripts.
python -m zipapp app.pyz -o newapp.pyz [-p interpreter]
Copy app.pyz to newapp.pyz, modifying the shebang line based
on the -p option (as for creating an archive, no -p option
means remove the shebang line). Specifying a destination is
mandatory.
In-place modification of an archive is *not* supported, as the
risk of damaging archives is too great for a simple tool.
As noted, the archives are standard zip files, and so can be unpacked
using any standard ZIP utility or Python's zipfile module. For this
reason, no interfaces to list the contents of an archive, or unpack
them, are provided or needed.
FAQ
---
Are you sure a standard ZIP utility can handle ``#!`` at the beginning?
Absolutely. The zipfile specification allows for arbitrary data to
be prepended to a zipfile. This feature is commonly used by
"self-extracting zip" programs. If your archive program can't
handle this, it is a bug in your archive program.
Isn't zipapp just a very thin wrapper over the zipfile module?
Yes. If you prefer to build your own Python zip application
archives using other tools, they will work just as well. The
zipapp module is a convenience, nothing more.
Why not use just use a ``.zip`` or ``.py`` extension?
Users expect a ``.zip`` file to be opened with an archive tool, and
expect a ``.py`` file to contain readable text. Both would be
confusing for this use case.
How does this compete with existing package formats?
The sdist, bdist and wheel formats are designed for packaging of
modules to be installed into an existing Python installation.
They are not intended to be used without installing. The
executable zip format is specifically designed for standalone use,
without needing to be installed. They are in effect a multi-file
version of a standalone Python script.
Rejected Proposals
==================
Convenience Values for Shebang Lines
------------------------------------
Is it worth having "convenience" forms for any of the common
interpreter values? For example, ``-p 3`` meaning the same as ``-p
"/usr/bin/env python3"``. It would save a lot of typing for the
common cases, as well as giving cross-platform options for people who
don't want or need to understand the intricacies of shebang handling
on "other" platforms.
Downsides are that it's not obvious how to translate the
abbreviations. For example, should "3" mean "/usr/bin/env python3",
"/usr/bin/python3", "python3", or something else? Also, there is no
obvious short form for the key case of "/usr/bin/env python" (any
available version of Python), which could easily result in scripts
being written with overly-restrictive shebang lines.
Overall, this seems like there are more problems than benefits, and as
a result has been dropped from consideration.
Registering ``.pyz`` as a Media Type
------------------------------------
It was suggested [3]_ that the ``.pyz`` extension should be registered
in the Unix database of extensions. While it makes sense to do this
as an equivalent of the Windows installer registering the extension,
the ``.py`` extension is not listed in the media types database [4]_.
It doesn't seem reasonable to register ``.pyz`` without ``.py``, so
this idea has been omitted from this PEP. An interested party could
arrange for *both* ``.py`` and ``.pyz`` to be registered at a future
date.
Default Interpreter
-------------------
The initial draft of this PEP proposed using ``/usr/bin/env python``
as the default interpreter. Unix users have problems with this
behaviour, as the default for the python command on many distributions
is Python 2, and it is felt that this PEP should prefer Python 3 by
default. However, using a command of ``python3`` can result in
unexpected behaviour for Windows users, where the default behaviour of
the launcher for the command ``python`` is commonly customised by users,
but the behaviour of ``python3`` may not be modified to match.
As a result, the principle "in the face of ambiguity, refuse to guess"
has been invoked, and archives have no shebang line unless explicitly
requested. On Windows, the archives will still be run (with the
default Python) by the launcher, and on Unix, the archives can be run
by explicitly invoking the desired Python interpreter.
Command Line Tool to Manage Shebang Lines
-----------------------------------------
It is conceivable that users would want to modify the shebang line for
an existing archive, or even just display the current shebang line.
This is tricky to do so with existing tools (zip programs typically
ignore prepended data totally, and text editors can have trouble
editing files containing binary data).
The zipapp module provides functions to handle the shebang line, but
does not include a command line interface to that functionality. This
is because it is not clear how to provide one without the resulting
interface being over-complex and potentially confusing. Changing the
shebang line is expected to be an uncommon requirement.
Reference Implementation
========================
A reference implementation is at http://bugs.python.org/issue23491.
References
==========
.. [1] Allow interpreter to execute a zip file
(http://bugs.python.org/issue1739468)
.. [2] Feature is not documented
(http://bugs.python.org/issue17359)
.. [3] Discussion of adding a .pyz mime type on python-dev
(https://mail.python.org/pipermail/python-dev/2015-February/138338.html)
.. [4] Register of media types
(http://www.iana.org/assignments/media-types/media-types.xhtml)
.. [5] pyzzer - A tool for creating Python-executable archives
(https://pypi.python.org/pypi/pyzzer)
.. [6] pex - The PEX packaging toolchain
(https://pypi.python.org/pypi/pex)
The discussion of this PEP took place on the python-dev mailing list,
in the thread starting at
https://mail.python.org/pipermail/python-dev/2015-February/138277.html
Copyright
=========
This document has been placed into the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: