py_shm (erlang_python v5.0.0)
View SourceShared memory regions between Erlang and Python contexts.
A region is a fixed-size file mapped MAP_SHARED through iommap. Erlang reads it with no copy (binary/3) and writes with one copy (write/3); Python maps the same file and sees it as a buffer (erlang.SharedMemory), in every context mode. The handle is a plain term, {$py_shm', Id, Path, Size}', so it travels inside any call argument or result.
{ok, Shm} = py_shm:new(64 * 1024 * 1024),
ok = py_shm:write(Shm, 0, Data),
{ok, Sum} = py_context:call(Ctx, myapp, sum_floats, [Shm]),
Out = py_shm:binary(Shm, 0, 1024),
ok = py_shm:close(Shm).iommap is an optional dependency: add {iommap, "1.1.3"} to your deps. Without it new/1,2 returns {error, iommap_not_available}.
The module also backs shared py_buffers (py_buffer:new(#{shared => true})): a region used as a ring, with the write position and the closed flag in a header page and flow control through the _py_buffer_wait and _py_buffer_consumed callbacks the Python side calls.
Owns: the region table (ETS), the backing files, and the ring state of shared buffers. Talks to: iommap (through apply/3, optional dependency), py_buffer (shared variant), py_callback (registers _py_buffer_wait, _py_buffer_consumed, _py_buffer_state). Never: maps memory into Python itself; that is _erlang_impl/_shm.py in each interpreter.
Summary
Functions
Whether iommap is available: shared memory needs it.
Create a shared streaming buffer (ring of RingSize bytes).
Append Data; blocks up to Timeout ms while the ring is full.
Close the region: the file is removed and the iommap handle closed. Python mappings stay valid until the wrapper is closed or collected.
Create a region of Size bytes owned by the calling process.
Create a region. Options: owner (pid whose exit closes the region, default the caller); writable (default true): with false Python maps the region read-only, so a buggy or hostile callee cannot change it.
Private directory for region files: /dev/shm when it exists (memory backed), else a 0700 directory under TMPDIR.
Copy Len bytes out of the region.
Read-only view of a handle for the Python side; Erlang keeps writing.
Copy Data into the region at Offset.
Types
-type buffer() :: {'$py_buffer', pos_integer(), binary(), pos_integer()}.
-type shm() :: {'$py_shm' | '$py_shm_ro', pos_integer(), binary(), non_neg_integer()}.
Functions
-spec available() -> boolean().
Whether iommap is available: shared memory needs it.
-spec binary(shm(), non_neg_integer(), non_neg_integer()) -> binary().
A binary over the region, no copy. It stays valid after close/1 (the mapping is kept as long as the binary is referenced). Its bytes are the region's: they change when Python writes, so treat it as a snapshot only once the callee is done with the handle, or use read/3 for a copy.
-spec buffer_close(buffer()) -> ok.
Create a shared streaming buffer (ring of RingSize bytes).
Append Data; blocks up to Timeout ms while the ring is full.
Close the region: the file is removed and the iommap handle closed. Python mappings stay valid until the wrapper is closed or collected.
-spec new(pos_integer()) -> {ok, shm()} | {error, term()}.
Create a region of Size bytes owned by the calling process.
-spec new(pos_integer(), map()) -> {ok, shm()} | {error, term()}.
Create a region. Options: owner (pid whose exit closes the region, default the caller); writable (default true): with false Python maps the region read-only, so a buggy or hostile callee cannot change it.
-spec private_dir() -> string().
Private directory for region files: /dev/shm when it exists (memory backed), else a 0700 directory under TMPDIR.
-spec read(shm(), non_neg_integer(), non_neg_integer()) -> {ok, binary()} | {error, term()}.
Copy Len bytes out of the region.
Read-only view of a handle for the Python side; Erlang keeps writing.
-spec size(shm()) -> non_neg_integer().
-spec write(shm(), non_neg_integer(), binary()) -> ok | {error, term()}.
Copy Data into the region at Offset.