Syntactic Permutations¶
direct import¶
valid:
import logging
import logging as log_mod
import logging as log_mod, sys
import logging.handlers
import logging.handlers as logh_mod
invalid:
import 5 #invalid identifier
import class #keyword
import (logging) #parentheses not allowed here
import (logging as log_mod) # not here either
indirect import¶
valid:
from logging import getLogger
from logging import handlers
from logging import handlers as logh_mod
from logging import (handlers)
from logging import (handlers as logh_mod)
invalid:
from logging import 5
relative indirect import¶
valid:
from . import spam
from . import spam as _spam
from .. import spam
from .spam import ham
Grammar (ASDL)¶
http://docs.python.org/dev/reference/grammar.html http://hg.python.org/cpython/file/default/Grammar/Grammar
import_stmt ::= import_name | import_from import_name ::= 'import' dotted_as_names import_from ::= ('from' (('.' | '...')* dotted_name | ('.' | '...')+)
This means...
Tokens and Keywords¶
http://docs.python.org/dev/library/token.html http://hg.python.org/cpython/file/default/Parser/tokenizer.c#l49 http://hg.python.org/cpython/file/default/Include/token.h
NAME
STAR
DOT
ELLIPSIS
COMMA
keywords:
import
from
as
For a while, import had 3 keywords all to itself!!!
AST¶
http://docs.python.org/dev/library/ast.html http://hg.python.org/cpython/file/default/Parser/Python.asdl http://hg.python.org/cpython/file/default/Include/Python-ast.h
Opcodes¶
http://docs.python.org/dev/library/dis.html http://hg.python.org/cpython/file/default/Include/opcode.h http://hg.python.org/cpython/file/default/Python/compile.c http://hg.python.org/cpython/file/default/Python/ceval.c
Examples of Looking under the Hood¶
Each of these examples will use the following code snippets:
import os
import os as os_mod
import os as os_mod, sys
import os.path:
def f(x): print(x) return x
Investigate Grammar Using tokenize Module¶
From ActiveState Recipe #...:
from io import BytesIO
from tokenize import tokenize
code = """def f(x):
print(x)
return x
"""
tokens = tokenize(BytesIO(code.encode('utf-8')).readline)
for toknum, tokval, _, _, _ in tokens:
print(toknum, tokval)
The __future__ Module¶
PEP 236 http://www.python.org/dev/peps/pep-0236/ http://docs.python.org/dev/library/__future__.html
feature | optional in | mandatory in | effect |
---|---|---|---|
nested_scopes | 2.1.0b1 | 2.2 | PEP 227: Statically Nested Scopes |
generators | 2.2.0a1 | 2.3 | PEP 255: Simple Generators |
division | 2.2.0a2 | 3.0 | PEP 238: Changing the Division Operator |
absolute_import | 2.5.0a1 | 2.7 | PEP 328: Imports: Multi-Line and Absolute/Relative |
with_statement | 2.5.0a1 | 2.6 | PEP 343: The “with” Statement |
print_function | 2.6.0a2 | 3.0 | PEP 3105: Make print a function |
unicode_literals | 2.6.0a2 | 3.0 | PEP 3112: Bytes literals in Python 3000 |
The Default import Handler¶
As already implied by sections 1.3 and 5.3, the import process is not as complex as you might expect. However, it is opaque enough that a thorough exposition would be worth it. Here is the entire process in one chunk of code:
...
- if a directory contains both a module file and a package directory, the package will be imported for the name and not the module.