Code map

View Source

Every source file, what it owns, and where to look for its behaviour. Status is live (on the path of a context created today) or test (only exercised by suites). Guides are in docs/, suites in test/. Start with architecture.

Erlang (src/)

ModuleOwnsStatusGuideSuites
pyPublic API facade: call/eval/exec, async helpers, memory, function registration; delegates streams, venvs and shared dictsliveREADME, getting-startedpy_SUITE, py_api_SUITE
py_streamGenerator streaming behind py:stream*livestreamingpy_stream_SUITE
py_venvVirtual environments behind py:ensure_venv and friendsliveREADME (venvs)py_venv_SUITE
py_shared_dictpy:shared_dict_* over the shared dict NIFsliveshared-dictpy_SUITE
py_contextThe API every mode answers (call/eval/exec, interrupt, kill, loops, pass_fd), the reply protocol and the pid to NIF reference table; init/4 hands the process to py_context_embedded or py_isolatedlivecontext-affinity, workers, interruptspy_context_SUITE, py_context_process_SUITE, py_interrupt_SUITE, py_worker_loop_SUITE
py_context_embeddedProcess body for worker and owngil mode: the receive loop, callbacks (suspension and pipe), worker loopslivearchitecture, state-machinessame
py_isolatedgen_statem driving a child process over the socket; restart policyliveisolatedpy_isolated_*_SUITE
py_context_routerPools and scheduler-affinity routinglivepools, context-affinitypy_context_router_SUITE, py_pool_SUITE
py_context_sup, py_context_initSupervisor of contexts; starts the default pool at bootlivepools(through the above)
py_nifErlang stubs and docs for every NIFliveapi-referenceall
py_callbackRegistry of Erlang funs callable as erlang.call('name', ...)liveREADME (callbacks)py_callback_encoding_SUITE, py_thread_callback_SUITE
py_thread_handlerCoordinator that gives each Python thread calling Erlang a handler process and a pipelivethreadingpy_thread_callback_SUITE, py_reentrant_SUITE
py_event_loopMain-interpreter asyncio loop: run, create_task, await, and the loop callbacks Python needsliveasynciopy_event_loop_SUITE, py_async_task_SUITE
py_event_loop_poolSeveral main-interpreter loops with process affinityliveasynciopy_event_loop_pool_SUITE
py_event_workerOne process per running loop receiving enif_select readiness and timersliveevent_loop_architecturepy_event_loop_SUITE, py_fd_ops_SUITE
py_event_worker_sup, py_event_worker_registrySupervisor and name registry of the event workersliveevent_loop_architecturepy_event_loop_SUITE
py_reactor_contextFD-owning context for the protocol-based reactorlivereactorpy_reactor_SUITE
py_channel, py_byte_channelTerm and byte queues between Erlang and Python coroutines (NIF resources)livechannelpy_channel_SUITE, py_byte_channel_SUITE
py_bufferNative streaming input buffer; shared variant delegates to py_shmlivebuffer, isolatedpy_buffer_SUITE, py_isolated_buffer_SUITE
py_shmShared memory regions over iommap and the ring behind shared buffersliveisolatedpy_isolated_shm_SUITE
py_importRegistry of imports and sys.path entries applied to every interpreterliveimportspy_import_SUITE
py_preloadCode run once per interpreter at startlivepreloadpy_preload_SUITE
py_stateShared key/value store visible from Python as erlang.state_get/set/delete/keysliveREADME (shared state)py_state_SUITE
py_semaphoreETS counting semaphore for rate limitinglivescalability(through py_SUITE)
py_logger, py_tracerPython logging into Erlang logger; tracing hooksliveloggingpy_logging_SUITE
erlang_python_app, erlang_python_supApplication start and the supervision treelivearchitectureall
py_utilSmall helperslive

C (c_src/)

py_nif.c is the only translation unit: it #includes the other .c files. Editing py_convert.c alone does not compile it alone; build with rebar3 compile. See c_src/README.md.

FileOwnsStatus
py_nif.hEvery shared type: py_context_t, request types, runtime state machine, atoms, globalslive
py_nif.cRuntime init, context creation and destruction, the request queue and the two context thread mains, the process-per-context NIFs (nif_context_*), process-local envs, py_ref, the NIF tablelive
py_convert.cpy_to_term / term_to_py, the type mapping, tagged tuples ({bytes, B}, shared handles)live
py_exec.cExecution mode detection and GIL helperslive
py_callback.cThe erlang Python module: call, send, whereis, Atom/Pid/Ref types, schedule markers, callback pipes, channel and shared dict methodslive
py_thread_worker.cPython threads calling Erlang through py_thread_handlerlive
py_subinterp_thread.cSub-interpreter thread pool used by owngil contexts and loop poolslive
py_event_loop.cErlangEventLoop support: enif_select readers/writers, timers, task injection, reactor dispatch, fd registry; also ~570 lines of test-only fd/TCP/UDP NIFslive; test section
py_channel.c, py_buffer.c, py_reactor_buffer.c, py_shared_dict.cThe corresponding resources and their Python-facing methodslive
py_logging.cLogging and tracing NIFslive
py_mem_limit.cPer-interpreter memory caps (owngil)live
py_util.c/.hMacros and helperslive
py_nif.h, py_event_loop.h, py_channel.h, py_buffer.h, py_reactor_buffer.h, py_subinterp_thread.h, py_util.hDeclarations shared between the included files; the struct comments in py_nif.h and py_event_loop.h carry the lock contractslive

The only code not on a live path is the "Test Helper Functions" section of py_event_loop.c (fd, pipe, TCP and UDP helpers the suites use).

Python (priv/)

priv/ is on sys.path of every interpreter. _erlang_impl is the Python half of the erlang module; the embedded C module delegates to it for the loop, channels and servers.

FileOwnsUsed by
_erlang_impl/__init__.pyPublic surface of erlang in embedded modes: run, sleep, spawn_task, loop policy, atom, channels, serverembedded
_erlang_impl/_loop.py, _policy.py, _transport.pyErlangEventLoop (uvloop-compatible) over enif_selectembedded
_erlang_impl/_reactor.pyProtocol-based reactor over fds Erlang ownsembedded
_erlang_impl/_channel.py, _byte_channel.pyPython side of channelsembedded
_erlang_impl/_server.pyserve, adopt, stop_serving on fds handed over by Erlang; plain asyncio, works in every modeall
_erlang_impl/_sandbox.py, _subprocess.pyAudit hook blocking fork/exec inside the VMembedded
_erlang_impl/_mode.pyDetects how Python is running (embedded, free-threaded, child)all
_erlang_impl/_etf.pyPure-Python ETF codec with the py_convert.c mappingisolated child
_erlang_impl/_isolated.pyChild runtime: socket frames, reader thread, re-entrant main loop, interrupt signal, asyncio loop, the erlang shimisolated child
_erlang_impl/_shm.pySharedMemory and SharedBuffer wrappers over mmapall
py_isolated_child.pyChild launcher: rlimits, parent-death signal, cgroup join, connectisolated child
test_erlang_loop.py, test_async_task.py, test_channel_ref.py, tests/Python-side tests of the loop, tasks and channelstest

Tests (test/)

Suites named py_<area>_SUITE. Cross-mode suites run the same cases in worker and isolated groups (py_isolated_SUITE, py_isolated_vm_SUITE, py_isolated_shm_SUITE, py_isolated_buffer_SUITE). Python helpers used by suites are test/py_test_*.py. test/coverage_audit.md maps public APIs to cases and every module to its suites; make check-code-map verifies this page and that table against the tree. test/test.config holds node-wide settings (memory limits flag).

Build and docs

rebar.config runs do_cmake.sh / do_build.sh (CMake in c_src/) as compile hooks; the NIF lands in priv/py_nif.so. make lint-docs checks that Erlang snippets in the guides call real exports and that Python snippets parse. rebar3 ex_doc builds the guides listed in rebar.config.