pythonimmediate package

Submodules

pythonimmediate.communicate module

class pythonimmediate.communicate.Communicator[source]

Bases: ABC

abstract send(data: bytes) → None[source]: Send data. This function is called in the textopy part.

abstract static setup() → tuple[pythonimmediate.communicate.Communicator, Callable[[], NoneType]][source]

Constructs an communicator object, that can be used to send information to this process.

The Communicator should be sent to the other process as part of the GlobalConfiguration object.

The ListenForwarder should be called in this process.

class pythonimmediate.communicate.GlobalConfiguration(debug: int = 0, communicator: Communicator = None, sanity_check_extra_line: bool = False, debug_force_buffered: bool = False, debug_log_communication: Optional[str] = None, naive_flush: bool = False)[source]

Bases: object

Represents the configuration.

This will be parsed from command-line argument in pytotex using from_args(), then sent to textopy side (which is where most of the processing is done) with a base64 of pickle encoding, on the textopy side a pseudo-config is created from the passed command-line arguments, then the real config is read in ParentProcessEngine’s constructor.

Which means it’s preferable to avoid any mutable state in the configuration object.

class pythonimmediate.communicate.MultiprocessingNetworkCommunicator(port: 'int', connection: 'Optional[Connection]' = None)[source]

Bases: Communicator

send(data: bytes) → None[source]: Send data. This function is called in the textopy part.

static setup() → tuple[pythonimmediate.communicate.MultiprocessingNetworkCommunicator, Callable[[], NoneType]][source]

Constructs an communicator object, that can be used to send information to this process.

The Communicator should be sent to the other process as part of the GlobalConfiguration object.

The ListenForwarder should be called in this process.

class pythonimmediate.communicate.UnnamedPipeCommunicator(pid: 'int', fileno: 'int', connection: 'Optional[IO[bytes]]' = None)[source]

Bases: Communicator

send(data: bytes) → None[source]: Send data. This function is called in the textopy part.

static setup() → tuple[pythonimmediate.communicate.UnnamedPipeCommunicator, Callable[[], NoneType]][source]

Constructs an communicator object, that can be used to send information to this process.

The Communicator should be sent to the other process as part of the GlobalConfiguration object.

The ListenForwarder should be called in this process.

pythonimmediate.copy_to_stderr module

module to copy everything from stdin to stderr.

needed to allow TeX to write to stderr.

pythonimmediate.engine module

Abstract engine class.

class pythonimmediate.engine.ChildProcessEngine(engine_name: Literal['pdftex', 'xetex', 'luatex'], args: Iterable[str] = (), env=None, autorestart: bool = False)[source]

Bases: Engine

An object that represents a $\TeX$ engine that runs as a subprocess of this process.

Can be used as a context manager to automatically close the subprocess when the context is exited. Alternatively close() can be used to manually terminate the subprocess.

For example, the following Python code, if run alone, will spawn a $\TeX$ process and use it to write “Hello world” to a file named a.txt in the temporary directory:

>>> from pythonimmediate.engine import ChildProcessEngine, default_engine
>>> from pythonimmediate import execute
>>> from pathlib import Path
>>> with ChildProcessEngine("pdftex") as engine, default_engine.set_engine(engine):
...     # do something with the engine, for example:
...     execute(r'''
...     \immediate\openout9=a.txt
...     \immediate\write9{Hello world}
...     \immediate\closeout9
...     ''')
...     (Path(engine.directory)/"a.txt").read_text()
'Hello world\n'
>>> # now the engine is closed and the file is cleaned up.
>>> (Path(engine.directory)/"a.txt").is_file()
False

Note that explicit engine argument must be passed in most functions. See DefaultEngine for a way to bypass that.

We attempt to do correct error detection on the Python side by parsing the output/log file:

>>> with ChildProcessEngine("pdftex") as engine, default_engine.set_engine(engine):
...     execute(r'\error')
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.

However, it’s not always easy. For example the following code will reproduce the output on error identically:

\message{^^J! error.^^J%
l.1 \errmessage{error}^^J%
^^J%
? }
\readline -1 to \xx

Parameters

args – List of additional arguments to be passed to the executable, such as --recorder etc.
env – See documentation of env argument in subprocess.Popen. Note that it’s usually recommended to append environment variables that should be set to os.environ instead of replacing it entirely.

autorestart –

Mostly for testing purpose – whenever an error happen, the engine will be killed and automatically restarted.

For example:

>>> from pythonimmediate import count
>>> engine=ChildProcessEngine("pdftex")
>>> with default_engine.set_engine(engine): execute(r'\error')
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.
>>> with default_engine.set_engine(engine): count[0]
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: error already happened

>>> engine=ChildProcessEngine("pdftex", autorestart=True)
>>> count[0]=2
>>> with default_engine.set_engine(engine): execute(r'\error')
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.
>>> with default_engine.set_engine(engine): count[0]
1
>>> with default_engine.set_engine(engine): execute(r'\error')
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.

class pythonimmediate.engine.DefaultEngine[source]

Bases: Engine, _local

A convenience class that can be used to avoid passing explicit engine argument to functions.

This is thread-safe, which means that each thread can have its own set default engine and set_engine() for one thread does not affect other threads.

Users should not instantiate this class directly. Instead, use default_engine.

Usage example:

default_engine.set_engine(engine)  # set only for this thread
execute("hello world")  # this is executed on engine=engine

See also

set_engine()

add_on_close(f: Callable[[], None]) → None[source]

Add a function that will be executed when the engine is closed.

>>> e=ChildProcessEngine("pdftex")
>>> e.add_on_close(lambda: print(1))
>>> e.add_on_close(lambda: print(2))
>>> e.close()
1
2

The same engine may be closed multiple times in case of autorestart, in that case the function will only be called once.

>>> e=ChildProcessEngine("pdftex", autorestart=True)
>>> a=[1, 2, 3]
>>> e.add_on_close(a.pop)
>>> from pythonimmediate import execute
>>> with default_engine.set_engine(e): execute(r"\error")
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.
>>> a
[1, 2]
>>> e.close()
>>> a
[1, 2]

property config: GlobalConfiguration: Self-explanatory.

engine: Optional[Engine]

Stores the engine being set internally.

Normally there’s no reason to access the internal engine directly, as self can be used like the engine inside.

get_engine() → Engine[source]

Convenience helper function, return the engine.

All the other functions that use this one (those that make use of the engine) will raise RuntimeError if the engine is None.

property name: Literal['pdftex', 'xetex', 'luatex']: Self-explanatory.

set_engine(engine: Optional[Engine]) → _SetDefaultEngineContextManager[source]

Set the default engine to another engine.

Can also be used as a context manager to revert to the original engine. Example:

>>> from pythonimmediate import execute
>>> _new_engine=ChildProcessEngine("pdftex")
>>> with default_engine.set_engine(_new_engine) as engine:  # only for this thread

… assert engine is _new_engine … assert default_engine.engine is _new_engine … execute(r”count0=5”) >>> # now the original engine is restored >>> _new_engine.close()

Note that the following form, while allowed, is discouraged because it may cause resource leak (the engine may keeps running even after the block exits depends on whether it’s garbage-collected):

>>> with default_engine.set_engine(ChildProcessEngine("pdftex")):
...     pass

It’s recommended to write the following instead, which will close e at the end of the block:

>>> with ChildProcessEngine("pdftex") as e, default_engine.set_engine(e):
...     pass

class pythonimmediate.engine.Engine[source]

Bases: ABC

add_on_close(f: Callable[[], None]) → None[source]

Add a function that will be executed when the engine is closed.

>>> e=ChildProcessEngine("pdftex")
>>> e.add_on_close(lambda: print(1))
>>> e.add_on_close(lambda: print(2))
>>> e.close()
1
2

The same engine may be closed multiple times in case of autorestart, in that case the function will only be called once.

>>> e=ChildProcessEngine("pdftex", autorestart=True)
>>> a=[1, 2, 3]
>>> e.add_on_close(a.pop)
>>> from pythonimmediate import execute
>>> with default_engine.set_engine(e): execute(r"\error")
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessError: Undefined control sequence.
>>> a
[1, 2]
>>> e.close()
>>> a
[1, 2]

close() → None[source]: Terminates the $\TeX$ subprocess gracefully.

property config: GlobalConfiguration: Self-explanatory.

property is_unicode: bool: Self-explanatory.

property name: Literal['pdftex', 'xetex', 'luatex']: Self-explanatory.

read() → bytes[source]

Internal function.

Read one line from the engine.

It must not be EOF otherwise there’s an error.

The returned line does not contain the newline character.

class pythonimmediate.engine.EngineStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Represent the status of the engine. Helper functions are supposed to manage the status by itself.

running: the engine is running $\TeX$ code and Python cannot write to it, only wait for Python commands.
waiting: the engine is waiting for Python to write a handler.
error: an error happened.
exited: the engine exited cleanly.

class pythonimmediate.engine.ParentProcessEngine(pseudo_config: GlobalConfiguration)[source]

Bases: Engine

Represent the engine if this process is started by the $\TeX$’s pythonimmediate library.

This should not be instantiated directly. Only pythonimmediate.textopy.main() should instantiate this.

exception pythonimmediate.engine.TeXProcessError[source]: Bases: RuntimeError

exception pythonimmediate.engine.TeXProcessExited[source]

Bases: Exception

An exception that will be raised if some operation makes the process exits.

>>> from pythonimmediate import execute, BalancedTokenList
>>> execute(r'\documentclass{article}\begin{document}hello world')
>>> execute(r'\end{document}')
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessExited
>>> execute(r'\documentclass{article}\begin{document}hello world')
>>> BalancedTokenList(r'\end{document}').execute()
Traceback (most recent call last):
    ...
pythonimmediate.engine.TeXProcessExited

pythonimmediate.engine.default_engine = <pythonimmediate.engine.DefaultEngine object>

A constant that can be used to avoid passing explicit engine argument to functions.

See documentation of DefaultEngine for more details.

For Python running inside a $\TeX$ process, useful attributes are name and is_unicode.

pythonimmediate.pytotex module

Receive things that should be passed to $\TeX$ from $\TeX$-to-Py half (pythonimmediate.textopy), then pass to $\TeX$.

User code are not executed here.

Anything that is put in pythonimmediatedebugextraargs environment variable will be appended to the command-line arguments. For example you could invoke $\TeX$ with pythonimmediatedebugextraargs='--debug-log-communication=/tmp/a.diff --debug=5' pdflatex test.tex to enable debugging facilities.

Side note --debug-log-communication also accept a $pid placeholder.

Supported command-line arguments:

usage: pytotex [-h] [-m {unnamed-pipe,multiprocessing-network}] [-d DEBUG]
               [--sanity-check-extra-line] [--no-sanity-check-extra-line]
               [--debug-log-communication DEBUG_LOG_COMMUNICATION]
               [--debug-force-buffered] [--naive-flush]

Named Arguments

-m, --mode

Possible choices: unnamed-pipe, multiprocessing-network

The mode of communication.

Refer to pythonimmediate.communicate for the detail on what each mode mean.

-d, --debug

Debug level. In [0..9].

Default: 0

--sanity-check-extra-line

Sanity check that there’s no extra line printed from [TeX] process. Should never be necessary unless the package is buggy. Might not work on Windows/MikTeX.

Default: False

--no-sanity-check-extra-line

Default: True

--debug-log-communication

Debug mode, log all communications. Pass the output path. For example you may want to specify --debug-log-communication=/tmp/a.diff to log all communication to a.diff file (because the lines are prefixed with < and >, diff syntax highlighting works nicely with it). Placeholder $pid is supported.

--debug-force-buffered

Debug mode, simulate [TeX] writes being 4096-byte buffered. Don’t use.

“This may raise the error message Fatal Python error: could not acquire lock for <_io.BufferedReader name='<stdin>'> at interpreter shutdown, possibly due to daemon threads because of how it’s implemented (a daemon thread read from stdin and forward to a pipe), but this feature is only used for debugging anyway so it does not matter.

Default: False

--naive-flush

Naively flush stdout by writing 4096 bytes to it when needed. Required in some [TeX] distribution that does not flush output.

Default: False

pythonimmediate.simple module

Simple interface, suitable for users who may not be aware of $\TeX$ subtleties, such as category codes.

This is only guaranteed to work properly in normal category code situations (in other words, \makeatletter or \ExplSyntaxOn are not officially supported). In all other cases, use the advanced API.

Nevertheless, there are still some parts that you must be aware of:

Any “output”, resulted from any command related to Python, can only be used to typeset text, it must not be used to pass “values” to other $\TeX$ commands.

This is already mentioned in the documentation of the $\TeX$ file in \py command, just repeated here.

For example:

\py{1+1}  % legal, typeset 2

\setcounter{abc}{\py{1+1}}  % illegal

% the following is still illegal no many how many ``\expanded`` and such are used
\edef\tmp{\py{1+1}}
\setcounter{abc}{\tmp}

\py{ r'\\setcounter{abc}{' + str(1+1) + '}' }  % legal workaround

Similar for commands defined with newcommand():

@newcommand
def test():
    print_TeX("123", end="")

Then the $\TeX$ code:

The result is \test.  % legal, typesets "123"
\setcounter{abc}{\test}  % illegal

There’s the function fully_expand(), but some $\TeX$ commands cannot be used inside this.

Refer to the Note on unexpandable and fragile macros section for more details.
Regarding values being passed from $\TeX$ to Python, you can do one of the following:
- Either simply do \py{value = r'''#1'''}, or
- \py{value = get_arg_str()}{#1}.
You can easily see both forms are equivalent, except that in the first form, #1 cannot end with unbalanced \ or contain triple single quotes.

Start with reading newcommand() and execute(). Then read the rest of the functions in this module.

pythonimmediate.simple.define_char(char: str) → Callable[[T2], T2][source]

Define a character to do some specific action.

Can be used as a decorator:

@define_char("×")
def multiplication_sign():
    print_TeX(end=r"\times")

Note

It’s not recommended to define commonly-used characters, for example if you define n then commands whose name contains n cannot be used anymore.

As another example, if you define -, then commands like \draw (1, 2) -- (3, 4); in TikZ cannot be used.

undefine_char() can be used to undo the effect.

pythonimmediate.simple.escape_str(s: str) → str[source]

Given an arbitrary string, output some [LaTeX] code that when executed will typeset the original string.

Example:

>>> escape_str("123")
'123'
>>> escape_str(r"a%#_^~\?")
'a\\%\\#\\_\\^{}\\~{}\\textbackslash{}?'
>>> escape_str("[")
'['
>>> escape_str("  ")
'\\ \\ '
>>> escape_str("\n")
'\\\\'

\n gets escaped to \\ so that multiple consecutive \n gives multiple line breaks.

< and > also needs to be escaped, but only in OT1 font encoding, see https://tex.stackexchange.com/q/2369/250119:

>>> escape_str("<>")
'\\textless{}\\textgreater{}'

Some Unicode characters may not be supported regardless.

Be careful with initial [ in text which may be interpreted as an optional argument in specific cases. For example https://tex.stackexchange.com/questions/34580/escape-character-in-latex/686889#comment1155297_34586.

pythonimmediate.simple.execute(block: str, engine: ~pythonimmediate.engine.Engine = <pythonimmediate.engine.DefaultEngine object>) → None[source]

Run a block of $\TeX$-code (might consist of multiple lines).

A simple example is execute('123') which simply typesets 123 (with a trailing newline, see the note in newcommand() documentation).

This is similar to print_TeX() but it’s executed immediately, and any error is immediately reported with Python traceback points to the caller.

On the other hand, because this is executed immediately, each part must be “self-contained”. With print_TeX() you can do the following:

print_TeX(r"\abc", end="")
print_TeX(r"def", end="")

and a single command \abcdef will be executed, but it does not work with this method.

As another consequence, all execute() are done before all print_TeX() in the same block. For example:

print_TeX(r"3")
execute(r"1")
print_TeX(r"4")
execute(r"2")

will typeset 1 2 3 4.

Note

For advanced users: it’s implemented with \scantokens, so catcode-changing commands are allowed inside.

pythonimmediate.simple.execute_tokenized(line: str, engine: ~pythonimmediate.engine.Engine = <pythonimmediate.engine.DefaultEngine object>) → None[source]: Temporary.

pythonimmediate.simple.f1(s: str, *, globals: Optional[dict] = None, locals: Optional[dict] = None, escape: Optional[Union[Tuple[str, ...], str]] = None) → str[source]

Helper function to construct a string from Python expression parts, similar to f-strings, but allow using arbitrary delimiters.

This is useful for constructing $\TeX$ code, because the { and } characters are frequently used in $\TeX$.

Example:

>>> a = 1
>>> b = 2
>>> f1("a=`a`, b=`b`")
'a=1, b=2'
>>> f1("`")
Traceback (most recent call last):
    ...
ValueError: Unbalanced delimiters!

The escape/delimiter character can be escaped by doubling it:

>>> f1("a``b")
'a`b'
>>> f1("a ` '````' ` b")
'a `` b'

It’s possible to customize the delimiters:

>>> f1("a=!a!", escape="!")
'a=1'

It’s possible to use different delimiters for the opening and the closing:

>>> f1("a=⟨a⟩, b=⟨b⟩", escape="⟨⟩")
'a=1, b=2'
>>> f1("<", escape="<>")
Traceback (most recent call last):
    ...
ValueError: Unbalanced delimiters!

There’s no way to escape them, they must be balanced in the string:

>>> f1("a=⟨ '⟨'+str(a)+'⟩' ⟩", escape="⟨⟩")
'a=⟨1⟩'

The default value of escape is stored in f1.default_escape, it can be reassigned:

>>> f1.default_escape="!"
>>> f1("a=!a!")
'a=1'
>>> f1.default_escape="`"

It’s even possible to use multiple characters as the delimiter:

>>> f1("a=**a**, b=**b**", escape=["**"])
'a=1, b=2'
>>> f1("a=[[a]], b=[[b]]", escape=["[[", "]]"])
'a=1, b=2'

In the first case above, remember to put the delimiter in a list because otherwise it will be interpreted as an opening and a closing delimiter.

Currently, format specifiers such as :d are not supported.

pythonimmediate.simple.fully_expand(content: str) → str[source]

Expand all macros in the given string.

Note on unexpandable and fragile macros

Not all $\TeX$ commands/macros can be used inside this function. If you try to, they will either return the input unchanged, or error out.

In such cases, the problem is impossible to solve, so just look for workarounds.

An example is the \py command, which is already mentioned in the initial introduction to the simple module:

>> fully_expand(r"\py{1+1}")
r"\py{1+1}"

Some other examples (assuming \usepackage{ifthen} is used):

>> execute(r'\ifthenelse{1=2}{equal}{not equal}')  # legal, typeset "not equal"
>> fully_expand(r'\ifthenelse{1=2}{equal}{not equal}')  # error out

>> execute(r'\uppercase{abc}')  # legal, typeset "ABC"
>> fully_expand(r'\uppercase{abc}')  # just returned verbatim
r'\uppercase {abc}'

There might be a few workarounds, but they’re all dependent on the exact macros involved.

For if-style commands, assign the desired result to a temporary variable:

>> execute(r'\ifthenelse{1=2}{\pyc{result = "equal"}}{\pyc{result = "not equal"}}')
>> result
"not equal"

For for-loop-style commands, you can do something similar (demonstrated here with tikz’s \foreach command):

>> result = []
>> execute(r'\foreach \x in {1, 2, ..., 10} {\pyc{result.append(var["x"])}}')
>> result
['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']

For macros such as \uppercase{...}, there’s no simple workaround. Reimplement it yourself (such as with .upper() in Python).

pythonimmediate.simple.get_arg_estr() → str[source]

Get a mandatory argument from the input stream, then process it as described in Note on argument expansion of estr-type functions.

Note on argument expansion of `estr`-type functions

These functions return a string, however they’re meant for strings other than $\TeX$ code.

The argument is fully expanded, and any “escaped character” are passed as the character itself to Python.

The behavior is similar to that of the \py command argument processing, see also the $\TeX$ package documentation.

Some examples: After the $\TeX$ code \def\myvalue{123abc} is executed, then:

if the argument in $\TeX$ code is {abc}, then the Python function will return "abc" (similar to as if get_arg_str() is used),
if the argument in $\TeX$ code is {\%a\#b\\\~}, then the Python function will return r"%a#b\~",
if the argument in $\TeX$ code is {\myvalue}, then the Python function will return "123abc".

pythonimmediate.simple.get_arg_str() → str[source]

Get a mandatory argument from the input stream.

It’s slightly difficult to explain what this does, so here’s an example:

@newcommand
def foo():
    a = get_arg_str()
    b = get_arg_str()
    c = get_arg_str()

The above defines a $\TeX$ function \foo, if it’s called in $\TeX$ code as:

\foo{xx}{yy}{zz}

then the variable a will be the string "xx", b will be "yy", and c will be "zz".

Note on parameter tokenization

Note

This function, as well as all the _str functions, return the argument that might be mangled in the following ways:

A space might be added after each command that consist of multiple characters
the ^^xx or ^^xxxx etc. notation is replaced by the character itself, or vice versa – literal tab character might get replaced with ^^I
multiple space characters may be collapsed into one
newline character may become space character
literally-typed \par in the source code may become double-newline character. (however you should never literally type \\par in the source code anyway he)
Even something like \\par in the source code may become double-newline character. (this is rare)

For example, \*\hell^^6F{ } may become the string r"\*\hello { }" in Python.

As such, verbatim-like commands in the argument are not supported. See get_verb_arg() for a workaround.

Nevertheless, if the argument consist of only “normal” $\TeX$ commands, re-executing the string should do the same thing as the original code.

In case of doubt, print out the string and check it manually.

Note

For advanced users:

This function corresponds to the m-type argument in xparse package.

It gets the argument, detokenize it, pass it through _replace_double_hash(), and return the result.

This is the simple API, as such it assumes normal category code values. Refer to BalancedTokenList.get_next() for a more advanced API.

pythonimmediate.simple.get_env_body_verb_approximate(envname: ~typing.Optional[str] = None, engine: ~pythonimmediate.engine.Engine = <pythonimmediate.engine.DefaultEngine object>) → tuple[str, str, int][source]

This function is to be used inside newenvironment()’s registered function in order to get the environment body.

It can only be used once, and it only works if the argument is not already tokenized, i.e., it appears at “top-level”/does not appear inside any command or some environments such as align. (basically where a \verb|...| command can appear.)

Returns: a tuple (body, filename, line_number) where body is the body of the environment, filename is the filename of the TeX file, and line_number is the line number of the last line of the environment.

Note

The returned code may have some tab characters converted to ^^I, trailing spaces stripped etc.

For advanced users: Basically it takes the following content until \end{⟨value of \@currenvir⟩}, as by the saveenv package.

pythonimmediate.simple.get_multiline_verb_arg() → str[source]: Get a multi-line verbatim argument. Usage is identical to get_verb_arg(), except newline characters in the argument is supported.

Note

in unusual category regime (such as that in \ExplSyntaxOn), it may return wrong result.

pythonimmediate.simple.get_next_char() → str[source]: Return the character of the following token as with peek_next_char(), but also removes it from the input stream.

pythonimmediate.simple.get_optional_arg_estr() → Optional[str][source]: Get an optional argument. See also Note on argument expansion of estr-type functions.

pythonimmediate.simple.get_optional_arg_str() → Optional[str][source]: Get an optional argument. See also Note on parameter tokenization.

Note

For advanced users: This function corresponds to the o-type argument in xparse package.

pythonimmediate.simple.get_verb_arg() → str[source]

Get a verbatim argument.

Similar to \verb, defined $\TeX$ commands that use this function can only be used at top level i.e. not inside any arguments.

This function behavior is identical to v-type argument in xparse package, you can use it like this:

\foo{xx%^\?}  % delimited with braces
\foo|xx%^\?|  % delimited with another symbol

See also

get_multiline_verb_arg() to support newline character in the argument.

Note

Hard TAB character in the argument gives an error until the corresponding LaTeX3 bug is fixed, see https://tex.stackexchange.com/q/508001/250119.

pythonimmediate.simple.is_balanced(content: str) → bool[source]

Check if the given string is balanced, i.e. if all opening and closing braces match.

This is a bit tricky to implement, it’s recommended to use the library function.

For example:

>>> is_balanced("a{b}c")
True
>>> is_balanced("a{b}c}")
False
>>> is_balanced(r"\{")
True

pythonimmediate.simple.newcommand(name: str, f: Optional[Callable]) → Callable[source]

Define a new $\TeX$-command.

The corresponding advanced API is Token.assign_func().

Example:

@newcommand
def myfunction():
    print_TeX("Hello world!", end="")

The above is mostly equivalent to \newcommand{\myfunction}{Hello world!}, it defines a $\TeX$ command \myfunction that prints Hello world!.

See also documentation of print_TeX() to understand the example above.

It can be used as either a decorator or a function:

def myfunction():
    print_TeX("Hello world!", end="")

newcommand("myfunction", myfunction)

An explicit command name can also be provided:

@newcommand("hello")
def myfunction():
    print_TeX("Hello world!", end="")

The above defines a $\TeX$ command \hello that prints Hello world!.

If name is not provided, it’s automatically deduced from the Python function name.

The above is not by itself very useful. Read the documentation of the following functions sequentially for a guide on how to define commands that take arguments:

get_arg_str()
get_arg_estr()
get_verb_arg()
get_optional_arg_str()
peek_next_char()
get_next_char()

Then see also (as natural extensions of the above):

get_multiline_verb_arg()
get_optional_arg_estr()

Note on trailing newline

Note

Regarding the end="" part in the example above, it’s used to prevent a spurious space, otherwise if you run the $\TeX$ code:

123\myfunction 456

it will be “equivalent” to:

123Hello world!
456

and the newline inserts a space.

Internally, the feature is implemented by appending % to the last line, as most of the time the code:

123Hello world!%
456

will be equivalent to:

123Hello world!456

but in unusual category code cases or cases of malformed $\TeX$ code (such as a trailing backslash at the end of the line), it may not be equivalent. Use the advanced API, or always include a final newline and manually add the comment character, if these cases happen.

pythonimmediate.simple.newenvironment(name: str, f: Optional[Callable]) → Callable[source]

Define a new $\TeX$ normal environment.

Note that the environment will normally not have access to the body of the environment, see newenvironment_verb() for some alternatives.

Parameters

name – the name of the environment, e.g. "myenv" or "myenv*".
f – a function that should execute the code for the begin part of the environment, yield, then execute the code for the end part of the environment.
engine – the engine to use.

Usage example:

@newenvironment("myenv")
def myenv():
    x=random.randint(1, 100)
    print_TeX(f"begin {x}")
    yield
    print_TeX(f"end {x}")

then the following $\TeX$ code:

\begin{myenv}
\begin{myenv}
Hello world!
\end{myenv}
\end{myenv}

might typeset the following content:

begin 42
begin 24
Hello world!
end 24
end 42

Functions such as get_arg_str() etc. can also be used in the first part of the function to take arguments.

If the name is omitted, the function name is used as the environment name.

It can be used either as a decorator or a function, see newcommand() for details.

pythonimmediate.simple.newenvironment_verb(name: str, f: Optional[Callable]) → Callable[source]

Define a new $\TeX$ environment that reads its body verbatim.

Note that space characters at the end of each line are removed.

The environment must not take any argument. For example the following built-in tabular environment takes some argument, so cannot be implemented with this function:

\begin{tabular}[t]
    ...
\end{tabular}

It can be used either as a decorator or a function, see newenvironment() for details.

Some usage example:

@newenvironment_verb("myenv")
def myenv(body: str):
    execute(body.replace("a", "b"))

If later the following $\TeX$ code is executed:

\begin{myenv}
aaa
\end{myenv}

then the value of the variable body will be "aaa\n", and the following content will be typeset:

bbb

Note

For advanced users: unlike a typical environment, the code will not be executed in a new $\TeX$ group.

pythonimmediate.simple.peek_next_char() → str[source]

Get the character of the following token, or empty string if it’s not a character.

This function can be used as follows to simulate e-type argument specifiers of xparse:

if peek_next_char()=="^":
    get_next_char()
    result = get_arg_str()
    # the following content in the input stream is ``^{...}`` and we stored the content inside to ``result``
else:
    pass  # there's no ``^...`` following in the input stream

It can also simulate s-type argument specifier (optional star):

if peek_next_char()=="*":
    get_next_char()
    # there's a star
else:
    pass  # there's no star

It can even simulate o-type argument specifier (optional argument delimited by [...]):

if peek_next_char()=="[":
    get_next_char()  # skip the `[`
    result=""
    while True:
        if peek_next_char():
            c=get_next_char()
            if c=="]": break
            result+=c
        else:
            # following in the input stream must be a control sequence, such as `\relax`
            result+=get_arg_str()
    # now result contains the content inside the `[...]`
else:
    pass  # there's no optional argument

Note that the above does not take into account the balance of braces or brackets, so:

If the following content in the input is [ab{cd]ef}gh] then the result will be ab{cd.
If the following content in the input is [ab[cd]ef] then the result will be ab[cd.

Note

For advanced users:

This function uses peek_next_meaning() under the hood to get the meaning of the following token. See that function documentation for a warning on undefined behavior.

Will also return nonempty if the next token is an implicit character token. This case is not supported and might give random error.

pythonimmediate.simple.print_TeX(*args, **kwargs) → None[source]

Unlike other packages, the normal print() function prints to the console.

This function can be used to print $\TeX$ code to be executed. For example:

\begin{pycode}
print_TeX("Hello world!")
\end{pycode}

It can also be used inside a custom-defined command, see the documentation of newcommand() for an example.

The signature is identical to that of print(). Passing explicit file= argument is not allowed.

Remark: normally the print() function prints out a newline with each invocation. A newline will appear as a spurious space in the output Use print_TeX(end="") to prevent this, see Note on trailing newline.

pythonimmediate.simple.put_next(arg: str) → None[source]

Put some content forward in the input stream.

Parameters

arg –

The content, must be a single line.

Note that there must not be any verbatim-like commands in the argument, so for example put_next(r"\verb|a|") is not allowed.

If there might be verbatim-like arguments, the problem is (almost) unsolvable. Refer to print_TeX() or execute() for workarounds, or use the advanced interface such as pythonimmediate.BalancedTokenList.put_next().

For example, if the following content in the input stream are {abc}{def}:

s = get_arg_str()  # s = "abc"
t = get_arg_str()  # t = "def"
put_next("{" + t + "}")
put_next("{" + s + "}")

After the above code, the content in the input stream is “mostly unchanged”.

Note

For advanced users: the argument is tokenized in the current category regime.

pythonimmediate.simple.put_next_tokenized(line: str, engine: ~pythonimmediate.engine.Engine = <pythonimmediate.engine.DefaultEngine object>) → None[source]: Temporary.

pythonimmediate.simple.renewcommand(name: str, f: Optional[Callable]) → Callable[source]: Redefine a $\TeX$-command. Usage is similar to newcommand().

pythonimmediate.simple.split_balanced(content: str, separator: str) → List[str][source]

Split the given string at the given substring, but only if the parts are balanced.

This is a bit tricky to implement, it’s recommended to use the library function.

If either content or separator is unbalanced, the function will raise ValueError.

For example:

>>> split_balanced("a{b,c},c{d}", ",")
['a{b,c}', 'c{d}']
>>> split_balanced(r"\{,\}", ",")
['\\{', '\\}']
>>> split_balanced("{", "{")
Traceback (most recent call last):
    ...
ValueError: Content is not balanced!

pythonimmediate.simple.strip_optional_braces(content: str) → str[source]

Strip the optional braces from the given string, if the whole string is wrapped in braces.

For example:

>>> strip_optional_braces("{a}")
'a'
>>> strip_optional_braces("a")
'a'
>>> strip_optional_braces("{a},{b}")
'{a},{b}'

pythonimmediate.simple.undefine_char(char: str) → None[source]: The opposite of define_char().

pythonimmediate.simple.var = VarManager()

Get and set value of “variables” in $\TeX$.

Can be used like this:

var["myvar"]="myvalue"  # "equivalent" to \def\myvar{myvalue}
var["myvar"]=r"\textbf{123}"  # "equivalent" to \def\myvar{\textbf{123}}
var["myvar"]=r"\def\test#1{#1}"  # "equivalent" to \tl_set:Nn \myvar {\def\test#1{#1}}  (note that `#` doesn't need to be doubled here unlike in `\def`)
var()["myvar"]="myvalue"  # pass explicit engine

# after ``\def\myvar{myvalue}`` (or ``\edef``, etc.) on TeX you can do:
print(var["myvar"])  # get the value of the variable, return a string

It’s currently undefined behavior if the passed-in value is not a “variable”. (in some future version additional checking might be performed in debug run)

Notes in Note on parameter tokenization apply – in other words, after you set a value it may become slightly different when read back.

pythonimmediate.texcmds module

This module defines the $\TeX$ commands and environments that will be available globally when the module is loaded.

The way the argument is absorbed is shown in the type annotation of the function, for example:

if the type is TTPELine or TTPEBlock then the argument is expanded according to Note on argument expansion of estr-type functions.
if the type is TTPLine or TTPBlock then the argument is not expanded, but the tokenization may change the form of the argument, refer to Note on parameter tokenization.

For example, the argument of py() has annotated type TTPEBlock, which means you can do the following:

\def\myvalue{1+2}
\py{\myvalue}

and the Python code 1+2 will be executed and 3 printed in $\TeX$.

Some command are not documented here, refer to:

add_handler() for \pythonimmediatecallhandler and \pythonimmediatelisten.
continue_until_passed_back() and continue_until_passed_back_str() for \pythonimmediatecontinue and \pythonimmediatecontinuenoarg.

pythonimmediate.texcmds.py(code: TTPEBlock) → None[source]

Evaluate some Python code provided as an argument and typeset the result as $\TeX$ code. For example \py{1+1} will typeset 2.

The code is evaluated as if with eval(), so assignments such as a=1 is not suitable here. Use pyc() instead.

pythonimmediate.texcmds.pyc(code: TTPEBlock) → None[source]

Execute some Python code provided as an argument, typeset whatever resulted by print_TeX().

The Python code is absorbed as described in Note on argument expansion of estr-type functions.

The code is evaluated as if with exec(), so assignments such as \pyc{a=1} can be used. Nevertheless, for long code :meth:` Use pyc() instead.

pythonimmediate.texcmds.pycode(code: TTPBlock, lineno_: TTPLine, filename: TTPLine, fileabspath: TTPLine) → None[source]

A $\TeX$ environment to execute some Python code. Must be used at top-level, and content can be verbatim.

If the code inside causes any error, the error traceback will point to the correct line in the $\TeX$ file.

Example:

\begin{pycode}
a = 1
b = 2
\end{pycode}

Note

Internal note:

The Python function itself is an auxiliary function.

The code is not executed immediately, instead we search for the source $\TeX$ file that contains the code (it must be found, otherwise an error is thrown), then read the source from that file.

$\TeX$ might mangle the code a bit before passing it to Python, as such we use can_be_mangled_to() (might return some false positive) to compare it with the code read from the sourcecode.

pythonimmediate.texcmds.pycodefuzzy(code: TTPBlock, lineno_: TTPLine) → None[source]: Same as pycode(), but may mangle the code (strip trailing spaces, etc. Refer to can_be_mangled_to() for technical details). Not recommended unless you don’t have [abspath]currfile package loaded.

pythonimmediate.texcmds.pycq(code: TTPEBlock) → None[source]: Similar to pyc(), however any output by print_TeX() is suppressed.

pythonimmediate.texcmds.pyfile(filepath: TTPELine) → None[source]

Execute a Python file from filepath, as if with pyc() or pycode().

Example: \pyfile{a/b.py}

pythonimmediate.texcmds.pyfilekpse(filename: TTPELine) → None[source]

Given a file name, use kpsewhich executable to search for its full path, then execute the file at that path.

This function is meant for library author wanting to execute Python code stored in separate file.

Example: \pyfilekpse{a.py} where a.py is stored along with the .sty files. Running kpsewhich a.py on the command-line should give the full path to it.

pythonimmediate.textopy module

pythonimmediate.textopy.main() → None[source]

This side does not receive the user-provided arguments directly, instead some parts of the configuration is decided by the pytotex side and forwarded to this half.

The arguments are re-parsed here anyway to provide a “temporary” configuration for the engine to work with before getting the real configuration.

Module contents

The main module. Contains Pythonic wrappers for much of $\TeX$’s API.

Refer to simple for the “simple” API – which allows users to avoid the need to know $\TeX$ internals such as category codes.

The fundamental data of $\TeX$ is a token, this is represented by Python’s Token object. A list of tokens is represented by TokenList object. If it’s balanced, BalancedTokenList should be used.

With that, you can manipulate the $\TeX$ input stream with BalancedTokenList.get_next(), BalancedTokenList.get_until(), TokenList.put_next().

Furthermore, executing $\TeX$ code is possible using continue_until_passed_back(). For example, the following code:

TokenList(r"\typeout{123}\pythonimmediatecontinuenoarg").put_next()
continue_until_passed_back()

will just use $\TeX$ to execute the code \typeout{123}.

With the 3 functions above, you can do everything that can be done in $\TeX$ (although maybe not very conveniently or quickly). Some other functions are provided, and for educational purposes, the way to implement it using the primitive functions are discussed.

expand_once(): TokenList(r"\expandafter\pythonimmediatecontinuenoarg").put_next(); continue_until_passed_back()
BalancedTokenList.expand_o(): TokenList(r"\expandafter\pythonimmediatecontinuenoarg\expandafter", self).put_next(); continue_until_passed_back(); return BalancedTokenList.get_next()

For example, if the current token list is test, the lines above will:
- put \expandafter\pythonimmediatecontinuenoarg\expandafter{\test} following in the input stream,
- pass control to $\TeX$,
- after one expansion step, the input stream becomes \pythonimmediatecontinuenoarg{⟨content of \test⟩},
- \pythonimmediatecontinuenoarg is executed, and execution is returned to Python,
- finally BalancedTokenList.get_next() gets the content of \test, as desired.
TokenList.execute(): (self+TokenList(r"\pythonimmediatecontinuenoarg")).put_next(); continue_until_passed_back()
NToken.put_next(): TokenList("\expandafter\pythonimmediatecontinuenoarg\noexpand\abc").put_next(); continue_until_passed_back() (as an example of putting a blue \abc token following in the input stream)
etc.

This is a table of $\TeX$ primitives, and their Python wrapper:

$TeX$	Python
`\let`	`Token.set_eq()`
`\ifx`	`Token.meaning_eq()`
`\meaning`	`NToken.meaning_str()`
`\futurelet`	`Token.set_future()`, `Token.set_future2()`
`\def`	`Token.tl()` (no parameter), `Token.set_func()` (define function to do some task)
`\edef`	`BalancedTokenList.expand_x()`
Get undelimited argument	`BalancedTokenList.get_next()`
Get delimited argument	`BalancedTokenList.get_until()`, `BalancedTokenList.get_until_brace()`
`\catcode`	`catcode`
`\count`	`count`, `Token.int()`
`\Umathcode`	`umathcode`
`\detokenize`	`BalancedTokenList.detokenize()`
`\begingroup`, `\endgroup`	`group`

In order to get a “value” stored in a “variable” (using expl3 terminology, this has various meanings e.g. a \countdef token, or a typical macro storing a token list), use a property on the token object itself:

Token.int() for \int_use:N \int_set:Nn,
Token.tl() for \tl_use:N \tl_set:Nn,
Token.str() for \str_use:N \str_set:Nn,
Token.bool(),
etc.

A token list can be:

interpreted as a string (provide it is already a string) using TokenList.str(),
converted from a Python string (opposite of the operation above) using TokenList.fstr(),
interpreted as an integer using TokenList.int(),
detokenized using BalancedTokenList.detokenize(),
expanded with BalancedTokenList.expand_x() or BalancedTokenList.expand_o(),
etc.

Some debug functionalities are provided and can be specified on the command-line, refer to pytotex documentation.

class pythonimmediate.BalancedTokenList(a: ~typing.Iterable = (), string_tokenizer: ~typing.Callable[[str], ~pythonimmediate.TokenList] = <bound method TokenList.e3 of <class 'pythonimmediate.TokenList'>>)[source]

Bases: TokenList

Represents a balanced token list.

Some useful methods to interact with $\TeX$ include expand_o(), expand_x(), get_next() and put_next(). See the corresponding methods’ documentation for usage examples.

See also Token list construction for shorthands to construct token lists in Python code.

Note

Runtime checking is not strictly enforced, use is_balanced() method explicitly if you need to check.

detokenize() → str[source]

Returns: a string, equal to the result of \detokenize applied to this token list.

execute() → None[source]

Execute this token list. It must not “peek ahead” in the input stream.

For example the token list \catcode1=2\relax can be executed safely (and sets the corresponding category code), but there’s no guarantee what will be assigned to \tmp when \futurelet\tmp is executed.

expand_estr() → str[source]

Expand this token list according to Note on argument expansion of estr-type functions.

It’s undefined behavior if the expansion result is unbalanced.

expand_o() → BalancedTokenList[source]

Return the o-expansion of this token list.

The result must be balanced, otherwise the behavior is undefined.

expand_x() → BalancedTokenList[source]

Return the x-expansion of this token list.

The result must be balanced, otherwise the behavior is undefined.

static get_next() → BalancedTokenList[source]: Get an (undelimited) argument from the $\TeX$ input stream.

static get_until(delimiter: BalancedTokenList, remove_braces: bool = True, long: bool = True) → BalancedTokenList[source]

Get a delimited argument from the $\TeX$ input stream, delimited by delimiter.

The delimiter itself will also be removed from the input stream.

Parameters: long – Works the same as \long primitive in $\TeX$ – if this is False then $\TeX$ fatal error Runaway argument will be raised if there’s a \par token in the argument.

static get_until_brace(long: bool = True) → BalancedTokenList[source]: Get a TokenList from the input stream delimited by {. The brace is not removed from the input stream.

put_next() → None[source]: Put this token list forward in the input stream.

class pythonimmediate.BlueToken(token: Token)[source]

Bases: NToken

Represents a blue token (see documentation of NToken).

property no_blue: Token: Return the result of this token after being “touched”, which drops its blue status if any.

property noexpand: BlueToken: Return the result of \noexpand applied on this token.

put_next() → None[source]: Put this token forward in the input stream.

pythonimmediate.C: alias of Catcode

class pythonimmediate.Catcode(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Enum, consist of begin_group, end_group, etc.

The corresponding enum value is the $\TeX$ code for the catcode:

>>> Catcode.letter.value
11

This class contains a shorthand to allow creating a token with little Python code. The individual Catcode objects can be called with either a character or a character code to create the object:

>>> C.letter("a")  # creates a token with category code letter and character code "a"=chr(97)
<Token: a₁₁>
>>> C.letter(97)  # same as above
<Token: a₁₁>

Both of the above forms are equivalent to CharacterToken(index=97, catcode=Catcode.letter).

See also Token list construction for more ways of constructing token lists.

property for_token: bool

Return whether a CharacterToken may have this catcode.

>>> Catcode.escape.for_token
False
>>> Catcode.letter.for_token
True

static lookup(x: int) → Catcode[source]

Construct from $\TeX$ code.

>>> C.lookup(11)
<Catcode.letter: 11>

class pythonimmediate.CharacterToken(index: int, catcode: Catcode)[source]

Bases: Token

Represent a character token. The preferred way to construct a character token is using C.

property assignable: bool: Whether this token can be assigned to i.e. it’s control sequence or active character.

property can_blue: bool: Return whether this token can possibly be blue i.e. expandable.

catcode: Catcode

>>> C.letter("a").catcode
<Catcode.letter: 11>

property chr: str

The character of this token.

>>> C.letter("a").chr
'a'

degree() → int[source]: return the imbalance degree for this token ({ -> 1, } -> -1, everything else -> 0)

index: int

The character code of this token.

>>> C.letter("a").index
97

serialize() → str[source]: Internal function, serialize this token to be able to pass to $\TeX$.

simple_detokenize(get_catcode: Callable[[int], Catcode]) → str[source]: Simple approximate detokenizer, implemented in Python.

str_code() → int[source]

self must represent a character of a $\TeX$ string. (i.e. equal to itself when detokenized)

Returns: the character code.

Note

See NTokenList.str_codes().

class pythonimmediate.ControlSequenceToken(csname: Union[str, bytes], is_unicode: Optional[bool] = None)[source]

Bases: Token

Represents a control sequence:

>>> ControlSequenceToken("abc")
<Token: \abc>

The preferred way to construct a control sequence is T.

Some care is needed to construct control sequence tokens whose name contains Unicode characters, as the exact token created depends on whether the engine is Unicode-based:

>>> with default_engine.set_engine(None):  # if there's no default_engine...
...     ControlSequenceToken("×")  # this will raise an error
Traceback (most recent call last):
    ...
AssertionError: Cannot construct a control sequence with non-ASCII characters without specifying is_unicode

The same control sequences may appear differently on Unicode and non-Unicode engines, and conversely, different control sequences may appear the same between Unicode and non-Unicode engines:

>>> a = ControlSequenceToken("u8:×", is_unicode=False)
>>> a
<Token: \u8:×>
>>> a == ControlSequenceToken(b"u8:\xc3\x97", is_unicode=False)
True
>>> a.codes
(117, 56, 58, 195, 151)
>>> b = ControlSequenceToken("u8:×", is_unicode=True)
>>> b
<Token: \u8:×>
>>> b.codes
(117, 56, 58, 215)
>>> a == b
False
>>> a == ControlSequenceToken("u8:\xc3\x97", is_unicode=True)
True

is_unicode will be fetched from default_engine if not explicitly specified.

property assignable: bool: Whether this token can be assigned to i.e. it’s control sequence or active character.

property codes: Tuple[int, ...]: Return the codes of this control sequence – that is, if \detokenize{...} is applied on this token, the tokens with the specified character codes (plus \escapechar) will result.

property csname: str: Return some readable name of the control sequence. Might return None if the name is not representable in UTF-8.

make = <pythonimmediate.ControlSequenceTokenMaker object>: Refer to the documentation of ControlSequenceTokenMaker.

serialize() → str[source]: Internal function, serialize this token to be able to pass to $\TeX$.

simple_detokenize(get_catcode: Callable[[int], Catcode]) → str[source]: Simple approximate detokenizer, implemented in Python.

class pythonimmediate.ControlSequenceTokenMaker(prefix: str)[source]

Bases: object

Shorthand to create ControlSequenceToken objects in Python easier.

>>> from pythonimmediate import T
>>> assert T is ControlSequenceToken.make
>>> T.hello
<Token: \hello>
>>> T["a@b"]  # for the "harder to construct" tokens
<Token: \a@b>
>>> P=ControlSequenceTokenMaker("__mymodule_")
>>> P.a
<Token: \__mymodule_a>

class pythonimmediate.MathClass(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]: Bases: Enum

class pythonimmediate.NToken[source]

Bases: ABC

Represent a possibly-notexpanded token. For convenience, a notexpanded token is called a blue token. It’s not always possible to determine the notexpanded status of a following token in the input stream.

Implementation note: Token objects must be frozen.

degree() → int[source]: return the imbalance degree for this token ({ -> 1, } -> -1, everything else -> 0)

meaning_eq(other: NToken) → bool[source]

Whether this token is the same in meaning as the token specified in the parameter other. Equivalent to $\TeX$’s \ifx.

Note that two tokens might have different meaning despite having equal meaning_str().

meaning_str(escapechar: Optional[Union[int, str]] = None) → str[source]

Get the meaning of this token as a string.

>>> C.other("-").meaning_str()
'the character -'
>>> T.relax.meaning_str(escapechar="?")
'?relax'
>>> T.relax.meaning_str()
'\\relax'

Note that all blue tokens have the meaning equal to \relax (or [unknown command code! (0, 1)] in a buggy LuaTeX implementation) with the backslash replaced by the current escapechar.

abstract property no_blue: Token: Return the result of this token after being “touched”, which drops its blue status if any.

abstract property noexpand: NToken: Return the result of \noexpand applied on this token.

abstract put_next()[source]: Put this token forward in the input stream.

str_code() → int[source]

self must represent a character of a $\TeX$ string. (i.e. equal to itself when detokenized)

Returns: the character code.

Note

See NTokenList.str_codes().

class pythonimmediate.NTokenList(a: ~typing.Iterable = (), string_tokenizer: ~typing.Callable[[str], ~pythonimmediate.TokenList] = <bound method TokenList.e3 of <class 'pythonimmediate.TokenList'>>)[source]

Bases: UserList

Similar to TokenList, but can contain blue tokens.

The class can be used identical to a Python list consist of NToken objects, plus some additional methods to operate on token lists.

Refer to the documentation of TokenList for some usage example.

execute() → None[source]: See BalancedTokenList.execute().

expand_x() → BalancedTokenList[source]: See BalancedTokenList.expand_x().

is_balanced() → bool[source]: Check if this is balanced.

put_next() → None[source]: See BalancedTokenList.put_next().

simple_parts() → List[Union[BalancedTokenList, Token, BlueToken]][source]

Internal function.

Split this NTokenList into a list of balanced non-blue parts, unbalanced {/} tokens, and blue tokens.

class pythonimmediate.PTTBalancedTokenList(data: 'BalancedTokenList')[source]

Bases: PyToTeXData

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PTTBlock(data: 'str')[source]

Bases: PyToTeXData

static coerce(s: str) → PTTBlock[source]: Construct a block from arbitrary string, delete some content if needed.

static ignore_last_space(s: str) → PTTBlock[source]: Construct a block from arbitrary string, deleting trailing spaces on each line.

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PTTInt(data: 'int')[source]

Bases: PyToTeXData

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PTTTeXLine(data: str)[source]

Bases: PyToTeXData

Represents a line to be tokenized in TeX’s current catcode regime. The trailing newline is not included, i.e. it’s tokenized under \endlinechar=-1.

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PTTVerbatimLine(data: 'str')[source]

Bases: PyToTeXData

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PTTVerbatimRawLine(data: bytes)[source]

Bases: PyToTeXData

Represents a line to be tokenized verbatim. Internally the \readline primitive is used, as such, any trailing spaces are stripped. The trailing newline is not included, i.e. it’s read under \endlinechar=-1.

read_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PyToTeXData[source]

Bases: ABC

Internal class (for now). Represent a data type that can be sent from Python to $\TeX$.

abstract static read_code(var: str) → str[source]

Takes an argument, the variable name (with backslash prefixed such as "\abc".)

Returns: some $\TeX$ code that when executed in expl3 category code regime,

will read a value of the specified data type and assign it to the variable.

abstract serialize() → bytes[source]: Return a bytes object that can be passed to engine.write() directly.

class pythonimmediate.PythonCallTeXFunctionType(*args, **kwargs)[source]: Bases: Protocol

class pythonimmediate.PythonCallTeXSyncFunctionType(*args, **kwargs)[source]: Bases: PythonCallTeXFunctionType, Protocol

class pythonimmediate.Python_call_TeX_data(TeX_code: 'str', recursive: 'bool', finish: 'bool', sync: 'Optional[bool]')[source]: Bases: object

class pythonimmediate.Python_call_TeX_extra(ptt_argtypes: 'Tuple[Type[PyToTeXData], ...]', ttp_argtypes: 'Union[Type[TeXToPyData], Tuple[Type[TeXToPyData], ...]]')[source]: Bases: object

pythonimmediate.Python_call_TeX_local(TeX_code: str, *, recursive: bool = True, sync: Optional[bool] = None, finish: bool = False) → Callable[source]: Internal function. See scan_Python_call_TeX().

class pythonimmediate.RedirectPrintTeX(t)[source]

Bases: object

A context manager. Use like this, where t is some file object:

with RedirectPrintTeX(t):
    pass  # some code

Then all print_TeX() function calls will be redirected to t.

pythonimmediate.T = <pythonimmediate.ControlSequenceTokenMaker object>: See ControlSequenceTokenMaker.

class pythonimmediate.TTPBalancedTokenList(a: ~typing.Iterable = (), string_tokenizer: ~typing.Callable[[str], ~pythonimmediate.TokenList] = <bound method TokenList.e3 of <class 'pythonimmediate.TokenList'>>)[source]

Bases: TeXToPyData, BalancedTokenList

static read() → TTPBalancedTokenList[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TTPBlock[source]

Bases: TeXToPyData, str

static read() → TTPBlock[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TTPEBlock[source]

Bases: TeXToPyData, str

A kind of argument that interprets “escaped string” and fully expand anything inside. For example, {\\} sends a single backslash to Python, {\{} sends a single { to Python.

Done by fully expand the argument in \escapechar=-1 and convert it to a string. Additional precaution is needed, see the note above (TODO write documentation).

Refer to Note on argument expansion of estr-type functions for more details.

static read() → TTPEBlock[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TTPELine[source]

Bases: TeXToPyData, str

Same as TTPEBlock, but for a single line only.

static read() → TTPELine[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TTPEmbeddedLine[source]

Bases: TeXToPyData, str

static read() → TTPEmbeddedLine[source]: Given that $\TeX$ has just sent the data, read into a Python object.

static send_code(self) → str[source]: Return some $\TeX$ code that sends the argument to Python, where arg represents a token list or equivalent (such as #1).

static send_code_var(self) → str[source]: Return some $\TeX$ code that sends the argument to Python, where var represents a token list variable (such as \l__my_var_tl) that contains the content to be sent.

class pythonimmediate.TTPLine[source]

Bases: TeXToPyData, str

static read() → TTPLine[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TTPRawLine[source]

Bases: TeXToPyData, bytes

static read() → TTPRawLine[source]: Given that $\TeX$ has just sent the data, read into a Python object.

send_code()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

send_code_var()

S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

class pythonimmediate.TeXToPyData[source]

Bases: ABC

Internal class (for now). Represent a data type that can be sent from $\TeX$ to Python.

abstract static read() → TeXToPyData[source]: Given that $\TeX$ has just sent the data, read into a Python object.

abstract static send_code(arg: str) → str[source]: Return some $\TeX$ code that sends the argument to Python, where arg represents a token list or equivalent (such as #1).

abstract static send_code_var(var: str) → str[source]: Return some $\TeX$ code that sends the argument to Python, where var represents a token list variable (such as \l__my_var_tl) that contains the content to be sent.

class pythonimmediate.Token[source]

Bases: NToken

Represent a $\TeX$ token, excluding the notexpanded possibility. See also documentation of NToken.

abstract property assignable: bool: Whether this token can be assigned to i.e. it’s control sequence or active character.

property blue: BlueToken: Return a BlueToken containing self. can_blue must be true.

bool() → bool[source]

Manipulate an expl3 bool variable.

>>> BalancedTokenList(r'\bool_set_true:N \l_tmpa_bool').execute()
>>> T.l_tmpa_bool.bool()
True

abstract property can_blue: bool: Return whether this token can possibly be blue i.e. expandable.

defined() → bool[source]: Return whether this token is defined, that is, its meaning is not undefined.

static deserialize(s: str | bytes) → Token[source]

See documentation of TokenList.deserialize().

Always return a single token.

static deserialize_bytes(data: bytes) → Token[source]

See documentation of TokenList.deserialize_bytes().

Always return a single token.

estr() → str[source]

Expand this token according to Note on argument expansion of estr-type functions.

It’s undefined behavior if the expansion result is unbalanced.

>>> T.l_tmpa_tl.tl(BalancedTokenList(r'ab\l_tmpb_tl'))
<BalancedTokenList: a₁₁ b₁₁ \l_tmpb_tl>
>>> T.l_tmpb_tl.tl(BalancedTokenList(r'cd123+$'))
<BalancedTokenList: c₁₁ d₁₁ 1₁₂ 2₁₂ 3₁₂ +₁₂ $₃>
>>> T.l_tmpa_tl.estr()
'abcd123+$'

..seealso::: BalancedTokenList.expand_estr()

static get_next() → Token[source]
static get_next(count: int) → TokenList: Get the following token.

Note

in LaTeX3 versions without the commit https://github.com/latex3/latex3/commit/24f7188904d6 sometimes this may error out.

Note

because of the internal implementation of \peek_analysis_map_inline:n, this may tokenize up to 2 tokens ahead (including the returned token), as well as occasionally return the wrong token in unavoidable cases.

int(val: Optional[int] = None) → int[source]

Manipulate an expl3 int variable.

>>> BalancedTokenList(r'\int_set:Nn \l_tmpa_int {5+6}').execute()
>>> T.l_tmpa_int.int()
11

See also

count.

is_expandable() → bool[source]

>>> T.relax.is_expandable()
False
>>> T.expandafter.is_expandable()
True
>>> T.undefined.is_expandable()
True
>>> BalancedTokenList([r'\protected\def\__protected_empty{}']).execute()
>>> T.__protected_empty.is_expandable()
True
>>> C.active("a").set_eq(T.empty)
>>> C.active("a").is_expandable()
True
>>> C.other("a").is_expandable()
False

property no_blue: Token: Return the result of this token after being “touched”, which drops its blue status if any.

property noexpand: NToken: Return the result of \noexpand applied on this token.

static peek_next() → Token[source]

Get the following token without removing it from the input stream.

Equivalent to get_next() then put_next() immediately. See documentation of get_next() for some notes.

put_next() → None[source]: Put this token forward in the input stream.

abstract serialize() → str[source]: Internal function, serialize this token to be able to pass to $\TeX$.

set_eq(other: NToken, global_: bool = False) → None[source]: Assign the meaning of this token to be equivalent to that of the other token.

set_func(f: Callable[[], None], global_: bool = False) → str[source]

Assign this token to call the Python function f when executed.

Returns an identifier, as described in add_handler().

set_future() → None[source]

Assign the meaning of this token to be equivalent to that of the following token in the input stream.

For example if this token is \a, and the input stream starts with bcde, then \a’s meaning will be assigned to that of the explicit character b.

Note

Tokenizes one more token in the input stream, and remove its blue status if any.

set_future2() → None[source]

Assign the meaning of this token to be equivalent to that of the second-next token in the input stream.

For example if this token is \a, and the input stream starts with bcde, then \a’s meaning will be assigned to that of the explicit character c.

Note

Tokenizes two more tokens in the input stream, and remove their blue status if any.

abstract simple_detokenize(get_catcode: Callable[[int], Catcode]) → str[source]: Simple approximate detokenizer, implemented in Python.

str(val: Optional[str] = None) → str[source]

Manipulate an expl3 str variable.

>>> BalancedTokenList(r'\str_set:Nn \l_tmpa_str {a+b}').execute()
>>> T.l_tmpa_str.str()
'a+b'
>>> T.l_tmpa_str.str('e+f')
'e+f'
>>> T.l_tmpa_str.str()
'e+f'
>>> T.l_tmpa_str.str('e+f\ng')
'e+f\ng'
>>> T.l_tmpa_str.str()
'e+f\ng'

tl(content: Optional[BalancedTokenList] = None, *, global_: bool = False) → BalancedTokenList[source]

Manipulate an expl3 tl variable.

>>> BalancedTokenList(r'\tl_set:Nn \l_tmpa_tl {1{2}}').execute()
>>> T.l_tmpa_tl.tl()
<BalancedTokenList: 1₁₂ {₁ 2₁₂ }₂>
>>> T.l_tmpa_tl.tl(BalancedTokenList('3+4'))
<BalancedTokenList: 3₁₂ +₁₂ 4₁₂>
>>> T.l_tmpa_tl.tl()
<BalancedTokenList: 3₁₂ +₁₂ 4₁₂>

class pythonimmediate.TokenList(a: ~typing.Iterable = (), string_tokenizer: ~typing.Callable[[str], ~pythonimmediate.TokenList] = <function TokenList_e3>)[source]

Bases: UserList

Represent a $\TeX$ token list, none of which can contain a blue token.

The class can be used identical to a Python list consist of Token objects, plus some additional methods to operate on token lists.

The list of tokens represented by this class does not need to be balanced. Usually you would want to use BalancedTokenList instead.

Token list construction

__init__() is the constructor of the class, and it accepts parameters in various different forms to allow convenient construction of token lists.

Most generally, you can construct a token list from any iterable consist of (recursively) iterables, or tokens, or strings. For example:

>>> TokenList([Catcode.letter("a"), "bc", [r"def\gh"]])
<TokenList: a₁₁ b₁₁ c₁₁ {₁ d₁₁ e₁₁ f₁₁ \gh }₂>

This will make a be the token list with value abc{def\gh }.

Note that the list that is recursively nested inside is used to represent a nesting level. A string will be “flattened” into the closest level, but a token list will not be flattened – they can be manually flattened with Python * syntax.

As a special case, you can construct from a string:

>>> TokenList(r"\let \a \b")
<TokenList: \let \a \b>

The constructor of other classes such as BalancedTokenList and NTokenList works the same way.

The above working implies that:

If you construct a token list from an existing token list, it will be copied (because a TokenList is a UserList of tokens, and iterating over it gives Token objects), similar to how you can copy a list with the list constructor:
```
>>> a = TokenList(["hello world"])
>>> b = TokenList(a)
>>> b
<TokenList: h₁₁ e₁₁ l₁₁ l₁₁ o₁₁ w₁₁ o₁₁ r₁₁ l₁₁ d₁₁>
>>> a==b
True
>>> a is b
False
```
Construct a token list from a list of tokens:
```
>>> TokenList([Catcode.letter("a"), Catcode.other("b"), T.test])
<TokenList: a₁₁ b₁₂ \test>
```
The above will define a to be ab\test, provided T is the object referred to in ControlSequenceTokenMaker.

See also Catcode for the explanation of the Catcode.letter("a") form.

By default, strings will be converted to token lists using TokenList.e3(), although you can customize it by:

Passing the second argument to the constructor.

Manually specify the type:

>>> TokenList([T.directlua, [*TokenList.fstr(r"hello%world\?")]])
<TokenList: \directlua {₁ h₁₂ e₁₂ l₁₂ l₁₂ o₁₂ %₁₂ w₁₂ o₁₂ r₁₂ l₁₂ d₁₂ \\₁₂ ?₁₂ }₂>

property balanced: BalancedTokenList

self must be balanced.

Returns: a BalancedTokenList containing the content of this object.

balanced_parts() → List[Union[BalancedTokenList, Token]][source]

Internal function, used for serialization and sending to $\TeX$.

Split this TokenList into a list of balanced parts and unbalanced {/} tokens.

check_balanced() → None[source]

ensure that this is balanced.

Raises: ValueError – if this is not balanced.

classmethod deserialize_bytes(data: bytes) → TokenListType[source]

Internal function.

Given a bytes object read directly from the engine, deserialize it.

classmethod doc(s: str) → TokenListType[source]

Approximate tokenizer in document (normal) catcode regime.

Refer to documentation of from_string() for details.

Usage example:

>>> BalancedTokenList.doc(r'\def\a{b}')
<BalancedTokenList: \def \a {₁ b₁₁ }₂>
>>> BalancedTokenList.doc('}')
Traceback (most recent call last):
    ...
ValueError: Token list <BalancedTokenList: }₂> is not balanced
>>> BalancedTokenList.doc('\n\n')
Traceback (most recent call last):
    ...
NotImplementedError: Double-newline to \par not implemented yet!
>>> TokenList.doc('}')
<TokenList: }₂>

classmethod e3(s: str) → TokenListType[source]

Approximate tokenizer in expl3 (\ExplSyntaxOn) catcode regime.

Refer to documentation of from_string() for details.

Usage example:

>>> BalancedTokenList.e3(r'\cs_new_protected:Npn \__mymodule_myfunction:n #1 { #1 #1 }')
<BalancedTokenList: \cs_new_protected:Npn \__mymodule_myfunction:n #₆ 1₁₂ {₁ #₆ 1₁₂ #₆ 1₁₂ }₂>
>>> BalancedTokenList.e3('a\tb\n\nc')
<BalancedTokenList: a₁₁ b₁₁ c₁₁>

execute() → None[source]

Execute this token list. It must not “peek ahead” in the input stream.

For example the token list \catcode1=2\relax can be executed safely (and sets the corresponding category code), but there’s no guarantee what will be assigned to \tmp when \futurelet\tmp is executed.

expand_x() → BalancedTokenList[source]

Return the x-expansion of this token list.

The result must be balanced, otherwise the behavior is undefined.

classmethod from_string(s: str, get_catcode: Callable[[int], Catcode], endlinechar: str) → TokenListType[source]

Approximate tokenizer implemented in Python.

Convert a string to a TokenList (or some subclass of it such as BalancedTokenList approximately.

This is an internal function and should not be used directly. Use one of e3() or doc() instead.

These are used to allow constructing a TokenList object in Python without being too verbose. Refer to Token list construction for alternatives.

The tokenization algorithm is slightly different from $\TeX$’s in the following respect:

multiple spaces are collapsed to one space, but only if it has character code space (32). i.e. in expl3 catcode, ~~ get tokenized to two spaces.
spaces with character code different from space (32) after a control sequence is not ignored. i.e. in expl3 catcode, ~ always become a space.
^^ syntax are not supported. Use Python’s escape syntax (e.g. ) as usual (of course that does not work in raw Python strings r"...").

Parameters: get_catcode – A function that given a character code, return its desired category code.

classmethod fstr(s: str, is_unicode: Optional[bool] = None) → TokenListType[source]

Approximate tokenizer in detokenized catcode regime.

Refer to documentation of from_string() for details. ^^J (or \n) is used to denote newlines.

>>> BalancedTokenList.fstr('hello world')
<BalancedTokenList: h₁₂ e₁₂ l₁₂ l₁₂ o₁₂  ₁₀ w₁₂ o₁₂ r₁₂ l₁₂ d₁₂>
>>> BalancedTokenList.fstr('ab\\c  d\n \t')
<BalancedTokenList: a₁₂ b₁₂ \\₁₂ c₁₂  ₁₀  ₁₀ d₁₂ \n₁₂  ₁₀ \t₁₂>

Some care need to be taken for Unicode strings. >>> with default_engine.set_engine(None): BalancedTokenList.fstr(‘α’) Traceback (most recent call last):

…

RuntimeError: Default engine not set for this thread! >>> with default_engine.set_engine(luatex_engine): BalancedTokenList.fstr(‘α’) <BalancedTokenList: α₁₂> >>> BalancedTokenList.fstr(‘α’) <BalancedTokenList: Î₁₂ ±₁₂>

int() → int[source]

Assume this token list contains an integer (as valid result of \number ...), returns the integer value.

At the moment, not much error checking is done.

is_balanced() → bool[source]: See NTokenList.is_balanced().

put_next() → None[source]: Put this token list forward in the input stream.

serialize_bytes() → bytes[source]

Internal function.

Given an engine, serialize it in a form that is suitable for writing directly to the engine.

str() → str[source]

self must represent a $\TeX$ string. (i.e. equal to itself when detokenized)

Returns: the string content.

>>> BalancedTokenList([C.other(0xce), C.other(0xb1)]).str()
'α'
>>> with default_engine.set_engine(luatex_engine): BalancedTokenList([C.other('α')]).str()
'α'

str_codes() → list[int][source]

self must represent a $\TeX$ string. (i.e. equal to itself when detokenized)

Returns: the string content.

>>> BalancedTokenList("abc").str_codes()
Traceback (most recent call last):
    ...
ValueError: this CharacterToken does not represent a string!
>>> BalancedTokenList("+-=").str_codes()
[43, 45, 61]

Note

In non-Unicode engines, each token will be replaced with a character with character code equal to the character code of that token. UTF-8 characters with character code >=0x80 will be represented by multiple characters in the returned string.

str_if_unicode(unicode: bool = True) → _Str[source]

Assume this token list represents a string in a (Unicode/non-Unicode) engine, return the string content.

If the engine is not Unicode, assume the string is encoded in UTF-8.

class pythonimmediate.Umathcode(family: int, cls: MathClass, position: int)[source]

Bases: object

Example of using active:

>>> Umathcode.parse(0x1000000)
Umathcode.active
>>> Umathcode.active.family
1

class pythonimmediate._CatcodeManager[source]

Bases: object

Python interface to manage the category code. Example usage:

>>> catcode[97]
<Catcode.letter: 11>
>>> catcode["a"] = C.letter

class pythonimmediate._CountManager[source]

Bases: object

Manipulate count registers. Interface is similar to catcode.

For example:

>>> count[5]=6  # equivalent to `\count5=6`
>>> count[5]
6
>>> count["endlinechar"]=10  # equivalent to `\endlinechar=10`
>>> T.endlinechar.int()  # can also be accessed this way
10
>>> count["endlinechar"]=13

As shown in the last example, accessing named count registers can also be done through Token.int().

class pythonimmediate._FrozenRelaxToken[source]

Bases: Token

>>> frozen_relax_token
<Token: [frozen]\relax>
>>> BalancedTokenList(r'\ifnum 0=0\fi').expand_x()
<BalancedTokenList: [frozen]\relax>

serialize() → str[source]: Internal function, serialize this token to be able to pass to $\TeX$.

simple_detokenize(get_catcode: Callable[[int], Catcode]) → str[source]: Simple approximate detokenizer, implemented in Python.

class pythonimmediate._GroupManager[source]

Bases: object

Create a semi-simple group.

Use as group.begin() and group.end(), or as a context manager:

>>> count[0]=5
>>> with group:
...     count[0]=6
...     count[0]
6
>>> count[0]
5

class pythonimmediate._UmathcodeManager[source]

Bases: object

For example:

>>> umathcode[0]
Traceback (most recent call last):
    ...
RuntimeError: umathcode is not available for non-Unicode engines!
>>> from pythonimmediate.engine import ChildProcessEngine
>>> with default_engine.set_engine(luatex_engine): umathcode["A"]
Umathcode(family=1, cls=<MathClass.variable_family: 7>, position=65 'A')

pythonimmediate.add_TeX_handler(t: BalancedTokenList, *, continue_included: bool = False) → str[source]

See call_TeX_handler().

Parameters

continue_included –

If this is set to True, \pythonimmediatecontinuenoarg token should be put when you want to return control to Python.

>>> with group: identifier=add_TeX_handler(BalancedTokenList(
...     r"\afterassignment\pythonimmediatecontinuenoarg \toks0="), continue_included=True)
>>> BalancedTokenList([["abc"]]).put_next()
>>> call_TeX_handler(identifier)  # this will assign \toks0 to be the following braced group
>>> toks[0]
<BalancedTokenList: a₁₁ b₁₁ c₁₁>

pythonimmediate.add_TeX_handler_param(t: BalancedTokenList, param: int | pythonimmediate.BalancedTokenList, *, continue_included: bool = False) → str[source]

Similar to add_TeX_handler(), however it will take parameters following in the input stream.

Parameters: continue_included – See add_TeX_handler().

>>> identifier=add_TeX_handler_param(BalancedTokenList(r"\def\l_tmpa_tl{#2,#1}"), 2)
>>> BalancedTokenList(r'{123}{456}').put_next()
>>> call_TeX_handler(identifier)
>>> T.l_tmpa_tl.tl()
<BalancedTokenList: 4₁₂ 5₁₂ 6₁₂ ,₁₂ 1₁₂ 2₁₂ 3₁₂>
>>> remove_TeX_handler(identifier)

pythonimmediate.add_handler(f: Callable[[], None], *, all_engines: bool = False) → str[source]

This function provides the facility to efficiently call Python code from $\TeX$ and without polluting the global namespace.

First, note that with pyc() you can do the following:

>>> a=get_user_scope()["a"]=[]
>>> execute(r"\def\test{\pyc{a.append(1)}}")

Then every time \test is executed on $\TeX$ side the corresponding Python code will be executed:

>>> a
[]
>>> execute(r"\test")
>>> a
[1]

However, this pollutes the Python global namespace as well as having to parse the string a.append(1) into Python code every time it’s called.

With this function, you can do the following:

>>> def myfunction(): execute(r"\advance\count0 by 1 ")  # it's possible to execute TeX code here
>>> identifier = add_handler(myfunction)
>>> execute(r"\def\test{\pythonimmediatecallhandler{" + identifier + r"}}")

>>> count[0]=5
>>> execute(r"\test")
>>> count[0]
6

The returned value, identifier, is a string consist of only English alphabetical letters, which should be used to pass into \pythonimmediatecallhandler $\TeX$ command and remove_handler().

The handlers must take a single argument of type Engine as input, and returns nothing.

pythonimmediate.add_handler_async(f: Callable[[], None], *, all_engines: bool = False) → str[source]

This function is for micro-optimization. Usage is not really recommended.

Similar to add_handler(), however, the function has these additional restrictions:

Within the function, it must not send anything to :math:`TeX`.
It must not cause a Python error, otherwise the error reporting facility may not work properly (does not print the correct $\TeX$ traceback).

Also, on the $\TeX$ side you need \pythonimmediatecallhandlerasync.

Example:

def myfunction():
    print(1)
identifier = add_handler(myfunction)
execute(r"\def\test{\pythonimmediatecallhandlerasync{" + identifier + "}}")

Note that in order to allow the Python function to call $\TeX$, it’s necessary to “listen” for callbacks on $\TeX$ side as well – as $\TeX$ does not have the capability to execute multiple threads, it’s necessary to explicitly listen for instructions from the Python side, which is what the command \pythonimmediatelisten in \pythonimmediatecallhandler’s implementation does.

Note

Internally, \pythonimmediatecallhandlerasync{abc} sends i⟨string⟩ from $\TeX$ to Python (optionally flushes the output), and the function with index ⟨string⟩ in this dict is called.

pythonimmediate.call_TeX_handler(identifier: str) → None[source]

Define some “handlers” in $\TeX$ that can be called quickly without re-sending the code every time it’s called.

Analog for add_handler(), remove_handler(), but on the $\TeX$ side.

The advantage is that it’s much faster than using BalancedTokenList.execute() every time. Otherwise the effect is identical.

Of course this is only for the current engine, and is global.

>>> identifier=add_TeX_handler(BalancedTokenList(r"\advance\count0 by 1"))
>>> count[0]=5
>>> count[0]
5
>>> call_TeX_handler(identifier)
>>> count[0]
6
>>> remove_TeX_handler(identifier)

pythonimmediate.catcode = <pythonimmediate._CatcodeManager object>: See _CatcodeManager.

pythonimmediate.continue_until_passed_back() → None[source]

Same as continue_until_passed_back_str() but nothing can be returned from $\TeX$ to Python.

So, this resumes the execution of $\TeX$ code until \pythonimmediatecontinuenoarg is executed.

See pythonimmediate for some usage examples.

pythonimmediate.continue_until_passed_back_str() → str[source]

Usage:

First put some tokens in the input stream that includes \pythonimmediatecontinue{...} (or %sync% \pythonimmediatelisten), then call continue_until_passed_back().

The function will only return when the \pythonimmediatecontinue is called.

pythonimmediate.count = <pythonimmediate._CountManager object>: See _CountManager.

pythonimmediate.expand_once() → None[source]

Expand the following content in the input stream once.

>>> BalancedTokenList(r'\iffalse 1 \else 2 \fi').put_next()  # now following tokens in the input stream is '\iffalse 1 \else 2 \fi'
>>> expand_once()  # now following tokens in the input stream is '2 \fi'
>>> Token.get_next()
<Token: 2₁₂>
>>> Token.get_next()
<Token: \fi>
>>> BalancedTokenList(r'\fi').execute()

pythonimmediate.frozen_relax_token = <Token: [frozen]\relax>: Constant representing the frozen \relax token. See _FrozenRelaxToken.

pythonimmediate.get_user_scope() → Dict[str, Any][source]

This is the global namespace where codes in py(), pyc(), pycode() etc. runs in. Mainly useful for ChildProcessEngine or cases when the scope is not the global scope (e.g. pyfilekpse()) only.

>>> aaa=1
>>> execute(r'\pyc{aaa}')
Traceback (most recent call last):
    ...
NameError: name 'aaa' is not defined
>>> get_user_scope()["aaa"]=1
>>> execute(r'\pyc{aaa}')

pythonimmediate.group = <pythonimmediate._GroupManager object>: See _GroupManager.

pythonimmediate.lua_try_eval(s: str) → Optional[str][source]

Evaluate some Lua code, if fail then execute it. Works like an interactive shell, first try to evaluate it as an expression, if fail execute it.

If you use IPython shell/Jupyter notebook, it may be desired to add a magic command to execute Lua code. For example in IPython: Create a file .ipython/profile_default/startup/lua_magic.py:

# Support %l <code> and %%l <newline> <line(s) of code> to execute Lua code in the LuaTeX engine.
from typing import Optional
from pythonimmediate import lua_try_eval
from IPython.core.magic import register_line_magic, register_cell_magic
register_line_magic("l")(lambda line: lua_try_eval(line))
@register_cell_magic("l")
def _cell_magic(line: str, cell: str)->Optional[str]:
    assert not line.strip(), "first line after %%l must be empty!"
    return lua_try_eval(cell)

>>> c=default_engine.set_engine(ChildProcessEngine("luatex"))
>>> lua_try_eval("2+3")
'5'
>>> lua_try_eval("do local a=2; return a+4 end")
'6'
>>> lua_try_eval("do local a=2 end")
>>> c.restore()

pythonimmediate.peek_next_meaning() → str[source]

Get the meaning of the following token, as a string, using the current \escapechar.

This is recommended over peek_next_token() as it will not tokenize an extra token.

It’s undefined behavior if there’s a newline (\newlinechar or ^^J, the latter is OS-specific) in the meaning string.

>>> BalancedTokenList("2").put_next()
>>> peek_next_meaning()
'the character 2'
>>> Token.get_next()
<Token: 2₁₂>

pythonimmediate.remove_TeX_handler(identifier: str) → None[source]: See call_TeX_handler().

pythonimmediate.remove_handler(identifier: str, *, all_engines: bool = False) → None[source]

Remove a handler with the given identifier.

Note that even if the corresponding $\TeX$ command is deleted, the command might have been copied to another command, so use this function with care.

See also

add_handler().

pythonimmediate.toks = <pythonimmediate._ToksManager object>: See _ToksManager.

pythonimmediate.typeout(s: str) → None[source]: Wrapper around LaTeX’s \typeout.

pythonimmediate.umathcode = <pythonimmediate._UmathcodeManager object>

Similar to catcode.

See _UmathcodeManager.

pythonimmediate.wlog(s: str) → None[source]: Wrapper around LaTeX’s \wlog.

pythonimmediate package

Submodules

pythonimmediate.communicate module

pythonimmediate.copy_to_stderr module

pythonimmediate.engine module

pythonimmediate.pytotex module

Named Arguments

pythonimmediate.simple module

Note on unexpandable and fragile macros

Note on argument expansion of estr-type functions

Note on parameter tokenization

Note on trailing newline

pythonimmediate.texcmds module

pythonimmediate.textopy module

Module contents

Token list construction

Note on argument expansion of `estr`-type functions