What’s New In Python 3.6¶
Release: | 3.6.0a1 |
---|---|
Date: | June 03, 2016 |
This article explains the new features in Python 3.6, compared to 3.5.
For full details, see the Misc/NEWS file.
Note
Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.6 moves towards release, so it’s worth checking back even after reading earlier versions.
Summary – Release highlights¶
New syntax features:
- PEP 498: Formatted string literals
Windows improvements:
- The
py.exe
launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn’t specify a version (via command line arguments or a config file). Handling of shebang lines remains unchanged - “python” refers to Python 2 in that case.
New Features¶
PEP 498: Formatted string literals¶
Formatted string literals are a new kind of string literal, prefixed
with 'f'
. They are similar to the format strings accepted by
str.format()
. They contain replacement fields surrounded by
curly braces. The replacement fields are expressions, which are
evaluated at run time, and then formatted using the format()
protocol.
>>> name = "Fred"
>>> f"He said his name is {name}."
'He said his name is Fred.'
See PEP 498 and the main documentation at Formatted string literals.
PYTHONMALLOC environment variable¶
The new PYTHONMALLOC
environment variable allows to set the Python
memory allocators and/or install debug hooks.
It is now possible to install debug hooks on Python memory allocators on Python
compiled in release mode using PYTHONMALLOC=debug
. Effects of debug hooks:
- Newly allocated memory is filled with the byte
0xCB
- Freed memory is filled with the byte
0xDB
- Detect violations of Python memory allocator API. For example,
PyObject_Free()
called on a memory block allocated byPyMem_Malloc()
. - Detect write before the start of the buffer (buffer underflow)
- Detect write after the end of the buffer (buffer overflow)
- Check that the GIL is held when allocator
functions of
PYMEM_DOMAIN_OBJ
(ex:PyObject_Malloc()
) andPYMEM_DOMAIN_MEM
(ex:PyMem_Malloc()
) domains are called.
Checking if the GIL is held is also a new feature of Python 3.6.
See the PyMem_SetupDebugHooks()
function for debug hooks on Python
memory allocators.
It is now also possible to force the usage of the malloc()
allocator of
the C library for all Python memory allocations using PYTHONMALLOC=malloc
.
It helps to use external memory debuggers like Valgrind on a Python compiled in
release mode.
On error, the debug hooks on Python memory allocators now use the
tracemalloc
module to get the traceback where a memory block was
allocated.
Example of fatal error on buffer overflow using
python3.6 -X tracemalloc=5
(store 5 frames in traces):
Debug memory block at address p=0x7fbcd41666f8: API 'o'
4 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x02 *** OUCH
at tail+1: 0xfb
at tail+2: 0xfb
at tail+3: 0xfb
at tail+4: 0xfb
at tail+5: 0xfb
at tail+6: 0xfb
at tail+7: 0xfb
The block was made by call #1233329 to debug malloc/realloc.
Data at p: 1a 2b 30 00
Memory block allocated at (most recent call first):
File "test/test_bytes.py", line 323
File "unittest/case.py", line 600
File "unittest/case.py", line 648
File "unittest/suite.py", line 122
File "unittest/suite.py", line 84
Fatal Python error: bad trailing pad byte
Current thread 0x00007fbcdbd32700 (most recent call first):
File "test/test_bytes.py", line 323 in test_hex
File "unittest/case.py", line 600 in run
File "unittest/case.py", line 648 in __call__
File "unittest/suite.py", line 122 in run
File "unittest/suite.py", line 84 in __call__
File "unittest/suite.py", line 122 in run
File "unittest/suite.py", line 84 in __call__
...
(Contributed by Victor Stinner in issue 26516 and issue 26564.)
Other Language Changes¶
- None yet.
New Modules¶
- None yet.
Improved Modules¶
asyncio¶
Since the asyncio
module is provisional,
all changes introduced in Python 3.6 have also been backported to Python
3.5.x.
Notable changes in the asyncio
module since Python 3.5.0:
- The
ensure_future()
function and all functions that use it, such asloop.run_until_complete()
, now accept all kinds of awaitable objects. (Contributed by Yury Selivanov.) - New
run_coroutine_threadsafe()
function to submit coroutines to event loops from other threads. (Contributed by Vincent Michel.) - New
Transport.is_closing()
method to check if the transport is closing or closed. (Contributed by Yury Selivanov.) - The
loop.create_server()
method can now accept a list of hosts. (Contributed by Yann Sionneau.) - New
loop.create_future()
method to create Future objects. This allows alternative event loop implementations, such as uvloop, to provide a fasterasyncio.Future
implementation. (Contributed by Yury Selivanov.) - New
loop.get_exception_handler()
method to get the current exception handler. (Contributed by Yury Selivanov.) - New
timeout()
context manager to simplify timeouts handling code. (Contributed by Andrew Svetlov.) - New
StreamReader.readuntil()
method to read data from the stream until a separator bytes sequence appears. (Contributed by Mark Korenberg.) - The
loop.getaddrinfo()
method is optimized to avoid calling the systemgetaddrinfo
function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.)
contextlib¶
The contextlib.AbstractContextManager
class has been added to
provide an abstract base class for context managers. It provides a
sensible default implementation for __enter__() which returns
self
and leaves __exit__() an abstract method. A matching
class has been added to the typing
module as
typing.ContextManager
.
(Contributed by Brett Cannon in issue 25609.)
datetime¶
The datetime.strftime()
and
date.strftime()
methods now support ISO 8601 date
directives %G
, %u
and %V
.
(Contributed by Ashley Anderson in issue 12006.)
faulthandler¶
On Windows, the faulthandler
module now installs a handler for Windows
exceptions: see faulthandler.enable()
. (Contributed by Victor Stinner in
issue 23848.)
os¶
A new close()
method allows explicitly closing a
scandir()
iterator. The scandir()
iterator now
supports the context manager protocol. If a scandir()
iterator is neither exhausted nor explicitly closed a ResourceWarning
will be emitted in its destructor.
(Contributed by Serhiy Storchaka in issue 25994.)
pickle¶
Objects that need calling __new__
with keyword arguments can now be pickled
using pickle protocols older than protocol version 4.
Protocol version 4 already supports this case. (Contributed by Serhiy
Storchaka in issue 24164.)
readline¶
Added set_auto_history()
to enable or disable
automatic addition of input to the history list. (Contributed by
Tyler Crompton in issue 26870.)
rlcompleter¶
Private and special attribute names now are omitted unless the prefix starts with underscores. A space or a colon is added after some completed keywords. (Contributed by Serhiy Storchaka in issue 25011 and issue 25209.)
Names of most attributes listed by dir()
are now completed.
Previously, names of properties and slots which were not yet created on
an instance were excluded. (Contributed by Martin Panter in issue 25590.)
site¶
When specifying paths to add to sys.path
in a .pth file,
you may now specify file paths on top of directories (e.g. zip files).
(Contributed by Wolfgang Langner in issue 26587).
socketserver¶
Servers based on the socketserver
module, including those
defined in http.server
, xmlrpc.server
and
wsgiref.simple_server
, now support the context manager
protocol.
(Contributed by Aviv Palivoda in issue 26404.)
subprocess¶
subprocess.Popen
destructor now emits a ResourceWarning
warning
if the child process is still running. Use the context manager protocol (with
proc: ...
) or call explicitly the wait()
method to
read the exit status of the child process (Contributed by Victor Stinner in
issue 26741).
telnetlib¶
Telnet
is now a context manager (contributed by
Stéphane Wirtel in issue 25485).
typing¶
The typing.ContextManager
class has been added for
representing contextlib.AbstractContextManager
.
(Contributed by Brett Cannon in issue 25609.)
unittest.mock¶
The Mock
class has the following improvements:
- Two new methods,
Mock.assert_called()
andMock.assert_called_once()
to check if the mock object was called. (Contributed by Amit Saha in issue 26323.)
urllib.robotparser¶
RobotFileParser
now supports the Crawl-delay
and
Request-rate
extensions.
(Contributed by Nikolay Bogoychev in issue 16099.)
warnings¶
A new optional source parameter has been added to the
warnings.warn_explicit()
function: the destroyed object which emitted a
ResourceWarning
. A source attribute has also been added to
warnings.WarningMessage
(contributed by Victor Stinner in
issue 26568 and issue 26567).
When a ResourceWarning
warning is logged, the tracemalloc
is now
used to try to retrieve the traceback where the detroyed object was allocated.
Example with the script example.py
:
import warnings
def func():
return open(__file__)
f = func()
f = None
Output of the command python3.6 -Wd -X tracemalloc=5 example.py
:
example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'>
f = None
Object allocated at (most recent call first):
File "example.py", lineno 4
return open(__file__)
File "example.py", lineno 6
f = func()
The “Object allocated at” traceback is new and only displayed if
tracemalloc
is tracing Python memory allocations and if the
warnings
was already imported.
winreg¶
Added the 64-bit integer type REG_QWORD
.
(Contributed by Clement Rouault in issue 23026.)
zipfile¶
A new ZipInfo.from_file()
class method
allows making a ZipInfo
instance from a filesystem file.
A new ZipInfo.is_dir()
method can be used
to check if the ZipInfo
instance represents a directory.
(Contributed by Thomas Kluyver in issue 26039.)
The ZipFile.open()
method can now be used to
write data into a ZIP file, as well as for extracting data.
(Contributed by Thomas Kluyver in issue 26039.)
zlib¶
The compress()
function now accepts keyword arguments.
(Contributed by Aviv Palivoda in issue 26243.)
fileinput¶
hook_encoded()
now supports the errors argument.
(Contributed by Joseph Hackman in issue 25788.)
Optimizations¶
- The ASCII decoder is now up to 60 times as fast for error handlers
surrogateescape
,ignore
andreplace
(Contributed by Victor Stinner in issue 24870). - The ASCII and the Latin1 encoders are now up to 3 times as fast for the
error handler
surrogateescape
(Contributed by Victor Stinner in issue 25227). - The UTF-8 encoder is now up to 75 times as fast for error handlers
ignore
,replace
,surrogateescape
,surrogatepass
(Contributed by Victor Stinner in issue 25267). - The UTF-8 decoder is now up to 15 times as fast for error handlers
ignore
,replace
andsurrogateescape
(Contributed by Victor Stinner in issue 25301). bytes % args
is now up to 2 times faster. (Contributed by Victor Stinner in issue 25349).bytearray % args
is now between 2.5 and 5 times faster. (Contributed by Victor Stinner in issue 25399).- Optimize
bytes.fromhex()
andbytearray.fromhex()
: they are now between 2x and 3.5x faster. (Contributed by Victor Stinner in issue 25401). - Optimize
bytes.replace(b'', b'.')
andbytearray.replace(b'', b'.')
: up to 80% faster. (Contributed by Josh Snider in issue 26574). - Allocator functions of the
PyMem_Malloc()
domain (PYMEM_DOMAIN_MEM
) now use the pymalloc memory allocator instead ofmalloc()
function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and usemalloc()
for larger memory blocks. (Contributed by Victor Stinner in issue 26249). pickle.load()
andpickle.loads()
are now up to 10% faster when deserializing many small objects (Contributed by Victor Stinner in issue 27056).
Build and C API Changes¶
- New
Py_FinalizeEx()
API which indicates if flushing buffered data failed (issue 5319).
Deprecated¶
New Keywords¶
async
and await
are not recommended to be used as variable, class,
function or module names. Introduced by PEP 492 in Python 3.5, they will
become proper keywords in Python 3.7.
Deprecated Python modules, functions and methods¶
importlib.machinery.SourceFileLoader.load_module()
andimportlib.machinery.SourcelessFileLoader.load_module()
are now deprecated. They were the only remaining implementations ofimportlib.abc.Loader.load_module()
inimportlib
that had not been deprecated in previous versions of Python in favour ofimportlib.abc.Loader.exec_module()
.
Deprecated functions and types of the C API¶
- None yet.
Deprecated features¶
- The
pyvenv
script has been deprecated in favour ofpython3 -m venv
. This prevents confusion as to what Python interpreterpyvenv
is connected to and thus what Python interpreter will be used by the virtual environment. (Contributed by Brett Cannon in issue 25154.) - When performing a relative import, falling back on
__name__
and__path__
from the calling module when__spec__
or__package__
are not defined now raises anImportWarning
. (Contributed by Rose Ames in issue 25791.)
Deprecated Python behavior¶
- Raising the
StopIteration
exception inside a generator will now generate aDeprecationWarning
, and will trigger aRuntimeError
in Python 3.7. See PEP 479: Change StopIteration handling inside generators for details.
Removed¶
API and Feature Removals¶
inspect.getmoduleinfo()
was removed (was deprecated since CPython 3.3).inspect.getmodulename()
should be used for obtaining the module name for a given path.traceback.Ignore
class andtraceback.usage
,traceback.modname
,traceback.fullmodname
,traceback.find_lines_from_code
,traceback.find_lines
,traceback.find_strings
,traceback.find_executable_lines
methods were removed from thetraceback
module. They were undocumented methods deprecated since Python 3.2 and equivalent functionality is available from private methods.- The
tk_menuBar()
andtk_bindForTraversal()
dummy methods intkinter
widget classes were removed (corresponding Tk commands were obsolete since Tk 4.0).
Porting to Python 3.6¶
This section lists previously described changes and other bugfixes that may require changes to your code.
Changes in the Python API¶
- The format of the
co_lnotab
attribute of code objects changed to support negative line number delta. By default, Python does not emit bytecode with negative line number delta. Functions usingframe.f_lineno
,PyFrame_GetLineNumber()
orPyCode_Addr2Line()
are not affected. Functions decoding directlyco_lnotab
should be updated to use a signed 8-bit integer type for the line number delta, but it’s only required to support applications using negative line number delta. SeeObjects/lnotab_notes.txt
for theco_lnotab
format and how to decode it, and see the PEP 511 for the rationale. - The functions in the
compileall
module now return booleans instead of1
or0
to represent success or failure, respectively. Thanks to booleans being a subclass of integers, this should only be an issue if you were doing identity checks for1
or0
. See issue 25768. - Reading the
port
attribute ofurllib.parse.urlsplit()
andurlparse()
results now raisesValueError
for out-of-range values, rather than returningNone
. See issue 20059. - The
imp
module now raises aDeprecationWarning
instead ofPendingDeprecationWarning
. - The following modules have had missing APIs added to their
__all__
attributes to match the documented APIs:calendar
,csv
,ElementTree
,enum
,fileinput
,ftplib
,logging
,optparse
,subprocess
,tarfile
,threading
andwave
. This means they will export new symbols whenimport *
is used. See issue 23883. - When performing a relative import, if
__package__
does not compare equal to__spec__.parent
thenImportWarning
is raised. (Contributed by Brett Cannon in issue 25791.) - When a relative import is performed and no parent package is known, then
ImportError
will be raised. Previously,SystemError
could be raised. (Contributed by Brett Cannon in issue 18018.) - Servers based on the
socketserver
module, including those defined inhttp.server
,xmlrpc.server
andwsgiref.simple_server
, now only catch exceptions derived fromException
. Therefore if a request handler raises an exception likeSystemExit
orKeyboardInterrupt
,handle_error()
is no longer called, and the exception will stop a single-threaded server. (Contributed by Martin Panter in issue 23430.) spwd.getspnam()
now raises aPermissionError
instead ofKeyError
if the user doesn’t have privileges.- The
socket.socket.close()
method now raises an exception if an error (e.g. EBADF) was reported by the underlying system call. See issue 26685. - The decode_data argument for
smtpd.SMTPChannel
andsmtpd.SMTPServer
constructors is nowFalse
by default. This means that the argument passed toprocess_message()
is now a bytes object by default, andprocess_message()
will be passed keyword arguments. Code that has already been updated in accordance with the deprecation warning generated by 3.5 will not be affected.
Changes in the C API¶
PyMem_Malloc()
allocator family now uses the pymalloc allocator rather than systemmalloc()
. Applications callingPyMem_Malloc()
without holding the GIL can now crash. Set thePYTHONMALLOC
environment variable todebug
to validate the usage of memory allocators in your application. See issue 26249.Py_Exit()
(and the main interpreter) now override the exit status with 120 if flushing buffered data failed. See issue 5319.