py_context (erlang_python v5.0.0)
View SourceThe context process: one Python execution environment, one request at a time.
Every mode goes through this module. In worker and owngil mode the process holds a NIF context resource, forwards each request to the context thread in C (nif_context_call_async) and waits for its {py_result, Ref, Result}. In isolated mode init/4 hands the process to py_isolated, which speaks the same messages to a child OS process. Callers do not see the difference.
Callbacks
When Python calls erlang.call, the context thread blocks on the callback pipe and sends {erlang_callback, Id, Fun, Args} to this process, which runs the registered function and writes the reply frame back. Nested requests from the callback are served inline, so callbacks can call Python again to any depth.
Owns: the public API, the reply protocol ({MRef, Reply}, timeouts, interrupt on timeout) and the pid to NIF reference table. Talks to: py_context_embedded (the process body for worker and owngil mode), py_isolated (isolated mode), py_nif (interrupt). Never: runs Python on a scheduler thread; the context thread does.
Summary
Functions
Call a Python function with empty kwargs.
Call a Python function.
Call a Python function with timeout.
Call a Python function with a process-local environment.
Call a method on a Python object reference.
Information about the child of an isolated context: os_pid, python_version, executable, platform.
Create a process-local Python environment for this context.
Alias for stop/1 for API consistency.
Evaluate a Python expression with empty locals.
Evaluate a Python expression.
Evaluate a Python expression with timeout.
Evaluate a Python expression with a process-local environment.
Execute Python statements.
Execute Python statements with a process-local environment.
Get the interpreter ID for this context.
Get the NIF context reference from a context process. This is useful for calling low-level py_nif functions directly.
Interrupt Python code currently running in this context.
Check if this context is a subinterpreter.
Kill the child process of an isolated context with SIGKILL.
Event loop reference of this context, usable with py_nif:submit_task/7 and py_event_loop:create_task/4.
Create a new context with options map.
Hand a file descriptor to the child of an isolated context.
Start a new py_context process.
Start a new py_context process with options.
Run an ErlangEventLoop forever on the context's thread.
Stop a py_context process.
Stop a loop started with start_loop/1,2.
Schedule Module:Func(Args...) on the context's event loop and return at once with {ok, TaskRef}.
submit/4 followed by py_event_loop:await/2 (default timeout 5000 ms).
Convert a Python object reference to an Erlang term.
Types
-type context() :: pid().
-type context_mode() :: worker | owngil | isolated.
Functions
-spec call(context(), atom() | binary(), atom() | binary(), list()) -> {ok, term()} | {error, term()}.
Call a Python function with empty kwargs.
This is a convenience wrapper for call/5 that defaults Kwargs to #{}.
-spec call(context(), atom() | binary(), atom() | binary(), list(), map()) -> {ok, term()} | {error, term()}.
Call a Python function.
-spec call(context(), atom() | binary(), atom() | binary(), list(), map(), timeout()) -> {ok, term()} | {error, term()}.
Call a Python function with timeout.
-spec call(context(), atom() | binary(), atom() | binary(), list(), map(), timeout(), reference()) -> {ok, term()} | {error, term()}.
Call a Python function with a process-local environment.
-spec call_method(context(), reference(), atom() | binary(), list()) -> {ok, term()} | {error, term()}.
Call a method on a Python object reference.
Information about the child of an isolated context: os_pid, python_version, executable, platform.
Create a process-local Python environment for this context.
The environment is created inside the context's interpreter to ensure the correct memory allocator is used. This is critical for subinterpreters where each interpreter has its own memory allocator.
The returned EnvRef should be stored in the calling process's dictionary, keyed by interpreter ID.
-spec destroy(context()) -> ok.
Alias for stop/1 for API consistency.
Evaluate a Python expression with empty locals.
This is a convenience wrapper for eval/3 that defaults Locals to #{}.
Evaluate a Python expression.
Evaluate a Python expression with timeout.
-spec eval(context(), binary() | string(), map(), timeout(), reference()) -> {ok, term()} | {error, term()}.
Evaluate a Python expression with a process-local environment.
Execute Python statements.
Execute Python statements with a process-local environment.
-spec get_interp_id(context()) -> {ok, non_neg_integer()} | {error, term()}.
Get the interpreter ID for this context.
Get the NIF context reference from a context process. This is useful for calling low-level py_nif functions directly.
-spec interrupt(context()) -> ok | not_running.
Interrupt Python code currently running in this context.
Raises KeyboardInterrupt in the thread executing the context; the in-flight call returns {error, interrupted}. Callable from any process, including while the context process is blocked in a NIF.
Returns not_running if the context is idle, unknown, or the exception could not be delivered. Code blocked in a C call (time.sleep, a numpy kernel, a socket read) is only interrupted once that call returns.
Check if this context is a subinterpreter.
Returns true for subinterpreter mode, false for worker mode. In worker mode, process-local environments are used. In subinterpreter mode, each context has its own isolated namespace.
-spec kill(context()) -> ok | {error, not_isolated}.
Kill the child process of an isolated context with SIGKILL.
Total and immediate, whatever the child is doing (a C call, a numpy kernel, a blocked read). In-flight calls return {error, killed}. With restart => true (the default) a fresh child is started and the context stays usable, with its Python state gone. Embedded contexts (worker, owngil) cannot be killed: they return {error, not_isolated}.
Event loop reference of this context, usable with py_nif:submit_task/7 and py_event_loop:create_task/4.
owngil contexts have their own loop; worker contexts share the main interpreter's loop (py_event_loop:get_loop/0).
Create a new context with options map.
Options: - mode - Context mode (worker | owngil | isolated), default: worker - memory_limit - Cap in bytes on memory allocated by this context. Requires mode => owngil and the runtime started with enable_memory_limits; see py_nif:context_set_memory_limit/2 for what is counted.
-spec pass_fd(context(), non_neg_integer()) -> {ok, non_neg_integer()} | {error, term()}.
Hand a file descriptor to the child of an isolated context.
The fd is sent over the control socket (SCM_RIGHTS) and the number it got in the child is returned; use it with erlang.server.serve from a submitted coroutine. Get the fd with py:dup_fd/1 on a listening socket.
-spec start_link(pos_integer(), context_mode()) -> {ok, pid()} | {error, term()}.
Start a new py_context process.
The process creates a Python context based on the mode: - worker - Create a thread-state worker (main interpreter namespace) - owngil - Create a sub-interpreter with its own GIL (Python 3.14+) - isolated - Run CPython in a child OS process (see py_isolated)
The owngil mode creates a dedicated pthread for each context, allowing true parallel Python execution. Requires Python 3.14+. The isolated mode gives failure isolation: the child can be killed, capped with rlimits, and a crash in a C extension only takes the child down.
-spec start_link(pos_integer(), context_mode(), map()) -> {ok, pid()} | {error, term()}.
Start a new py_context process with options.
See new/1 for the recognised options.
Run an ErlangEventLoop forever on the context's thread.
Returns as soon as the loop is started. The loop keeps running until stop_loop/1,2 or interrupt/1; the owner (the caller by default) receives {py_loop_exit, Ctx, Result} when it ends, where Result is the return of the exec that ran it (ok, {error, interrupted}, or a Python error).
While the loop runs, call/eval/exec/call_method on this context return {error, loop_running}: the thread is busy in the loop, and a timed-out call would interrupt it. Use submit/4,5 and submit_await/4,5,6 instead; they inject coroutines into the running loop.
Options: - owner - pid that receives {py_loop_exit, Ctx, Result} (default: caller). If the owner dies the loop is stopped.
-spec stop(context()) -> ok.
Stop a py_context process.
Stop a loop started with start_loop/1,2.
Asks the loop to stop from inside (a coroutine calling loop.stop()), then interrupts the thread if it has not exited after Grace ms (default 5000). Returns ok once the loop has exited, {error, no_loop} when none is running, {error, timeout} if it survived the interrupt too.
-spec stop_loop(context(), non_neg_integer()) -> ok | {error, term()}.
-spec submit(context(), atom() | binary(), atom() | binary(), list()) -> {ok, reference()} | {error, term()}.
Schedule Module:Func(Args...) on the context's event loop and return at once with {ok, TaskRef}.
Works whether or not start_loop/1 is active: with a running loop the coroutine is injected into it, otherwise the event worker steps the loop. The result arrives as {async_result, TaskRef, {ok, Value} | {error, R}}; use py_event_loop:await/1,2 or submit_await/4,5,6. Coroutine functions are awaited, plain functions are called and their value returned. Module must be importable in the context (sys.modules), so put entry points in a module rather than in the exec namespace.
-spec submit_await(context(), atom() | binary(), atom() | binary(), list()) -> {ok, term()} | {error, term()}.
submit/4 followed by py_event_loop:await/2 (default timeout 5000 ms).
Convert a Python object reference to an Erlang term.