Gralkor.GraphitiPool (jido_gralkor v11.0.0)

Copy Markdown View Source

Per-group Graphiti instance cache, plus the gateway for graphiti operations.

Holds one shared AsyncFalkorDB (the embedded redis-server child lives here) and lazily constructs one Graphiti instance per group_id. Cached in ETS for concurrent reads — for/1 only hits the GenServer on a cache miss (i.e. the first time any caller asks for a given group). Once cached, thousands of callers can read the instance simultaneously without going through the GenServer.

Pythonx releases the GIL during graphiti's awaited I/O, so searches and remote operations parallelise naturally. Embedded add_episode calls alone use monitored admission through the GenServer because every group shares one locally owned Redis connection; searches do not enter that queue.

See test-trees/unit/graphiti-pool_TEST_TREES.md.

Summary

Functions

Ingest one episode (text content) into group_id via graphiti's add_episode. Auto-generates a unique episode name. When ontology is a module declared with use Gralkor.Ontology, its payload is materialised into graphiti's entity_types, edge_types, edge_type_map, and excluded_entity_types (cached per ontology module in the GenServer state).

The credential for provider, read on the Elixir side.

Build communities for group_id.

Rebuild indices and constraints across the whole graph.

Returns a specification to start this module under a supervisor.

Return the Graphiti instance for group_id, creating it on first use.

Returns one episode by its exact UUID.

Pure projection from an __ontology__/0 payload to the plain data handed across the Pythonx boundary. A graphiti add_episode kwarg (entity_types, edge_types, edge_type_map, excluded_entity_types) is populated iff its payload collection is present; the Pythonx side never re-decides inclusion, it materialises exactly what this spec carries. No Pythonx, no LLM — this is the deterministic contract the materialisation half trusts.

Remove an episode and its orphaned edges/nodes from the graph.

Run graphiti's hybrid EDGE search against group_id. Returns {:ok, [%{fact:, created_at:, valid_at:, invalid_at:, expired_at:}]} whose individual entries can be rendered by Gralkor.Format.format_fact/1.

Episode search against group_id via graphiti's search_ with an episode-only config. Returns {:ok, [%{content:, source_description:}]} — the episode bodies as they were written.

Node search against group_id via graphiti's search_ with the NODE_HYBRID_SEARCH_RRF recipe. Unlike search/4 (which returns edges and whose node_labels filter matches edges by endpoint), this returns nodes that are not reliably reachable through edge search.

Decide which provider builds each shared client, from the two configured model specs. Pure — no Pythonx, no credentials read — so the per-role dispatch is pinned deterministically and default_construct_shared_clients/2 is left with nothing to decide.

Build a compact one-line reason from a Pythonx.Error.

Functions

add_episode(server \\ __MODULE__, group_id, content, source_description, ontology, opts \\ [])

@spec add_episode(
  GenServer.server(),
  String.t(),
  String.t(),
  String.t(),
  module() | nil,
  keyword()
) :: :ok | {:error, term()}

Ingest one episode (text content) into group_id via graphiti's add_episode. Auto-generates a unique episode name. When ontology is a module declared with use Gralkor.Ontology, its payload is materialised into graphiti's entity_types, edge_types, edge_type_map, and excluded_entity_types (cached per ontology module in the GenServer state).

Options

  • :uuid — optional deterministic episode UUID. A graph-backed renewable claim serializes that UUID across application runtimes. The episode, extracted entities and edges, and durable completion marker persist in one generation-fenced graph query. A missing UUID is created, an equal marked episode succeeds without extraction, an equal unmarked episode resumes extraction, and conflicting immutable episode content returns an episode conflict.
  • :lens — optional originating Lens name. It is appended to the episode's source description before the single graphiti add_episode call.
  • :source_kind:conversation, :document, or :structured_record, mapped to graphiti's message, text, or JSON episode type respectively.

api_key!(provider)

@spec api_key!(atom()) :: String.t()

The credential for provider, read on the Elixir side.

Erlang's os:putenv keeps its own table and never reaches the C environment, so a credential set from Elixir — a consumer's runtime.exs, or the test helper loading .env — is invisible to the embedded interpreter's os.environ. Every client constructor therefore takes its key as an explicit argument rather than letting the Python client read the variable itself.

Raises when the variable is absent; validate_native_models!/2 has already proven it present for every provider a role selects.

build_communities(server \\ __MODULE__, group_id)

@spec build_communities(GenServer.server(), String.t()) ::
  {:ok, %{communities: non_neg_integer(), edges: non_neg_integer()}}
  | {:error, term()}

Build communities for group_id.

build_indices(server \\ __MODULE__)

@spec build_indices(GenServer.server()) ::
  {:ok, %{status: String.t()}} | {:error, term()}

Rebuild indices and constraints across the whole graph.

Every group is its own FalkorDB database, so "the whole graph" is every group this pool holds an instance for — rebuilding one group's database would leave every other group untouched. A group whose instance has not been created yet needs no rebuild: initialise_instance/1 builds its indices the moment it is.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

for(server \\ __MODULE__, group_id)

@spec for(GenServer.server(), String.t()) :: any()

Return the Graphiti instance for group_id, creating it on first use.

Concurrent callers do not block each other once the instance is cached. Construction itself is serialised through the GenServer so two callers asking for the same group_id at the same time don't both construct it.

get_episode(server \\ __MODULE__, group_id, episode_uuid)

@spec get_episode(GenServer.server(), String.t(), String.t()) ::
  {:ok, map()} | {:error, :not_found | term()}

Returns one episode by its exact UUID.

graphiti_boundary_spec(map)

@spec graphiti_boundary_spec(map()) :: %{optional(atom()) => term()}

Pure projection from an __ontology__/0 payload to the plain data handed across the Pythonx boundary. A graphiti add_episode kwarg (entity_types, edge_types, edge_type_map, excluded_entity_types) is populated iff its payload collection is present; the Pythonx side never re-decides inclusion, it materialises exactly what this spec carries. No Pythonx, no LLM — this is the deterministic contract the materialisation half trusts.

remove_episode(server \\ __MODULE__, group_id, episode_uuid)

@spec remove_episode(GenServer.server(), String.t(), String.t()) ::
  :ok | {:error, term()}

Remove an episode and its orphaned edges/nodes from the graph.

Calls graphiti's remove_episode(uuid) which deletes the episode, its entity edges that were created by that episode, and any entity nodes referenced only by the deleted episode.

replace_graph(server \\ __MODULE__, group_id, lens_name, atom, map)

@spec replace_graph(
  GenServer.server(),
  String.t(),
  String.t(),
  :property_graph,
  Gralkor.Graph.property_graph()
) :: :ok | {:error, term()}

search(server \\ __MODULE__, group_id, query, max_results, opts \\ [])

@spec search(GenServer.server(), String.t(), String.t(), pos_integer(), keyword()) ::
  {:ok, [map()]} | {:error, term()}

Run graphiti's hybrid EDGE search against group_id. Returns {:ok, [%{fact:, created_at:, valid_at:, invalid_at:, expired_at:}]} whose individual entries can be rendered by Gralkor.Format.format_fact/1.

For retrieving custom-entity nodes, use search_nodes/5 — edge search's node-label filtering matches edges by endpoint and misses standalone nodes.

search_episodes(group_id, query, max_results)

@spec search_episodes(String.t(), String.t(), pos_integer()) ::
  {:ok, [map()]} | {:error, term()}

Episode search against group_id via graphiti's search_ with an episode-only config. Returns {:ok, [%{content:, source_description:}]} — the episode bodies as they were written.

This is the primitive for content Gralkor must read back verbatim. Edge and node search both return what an extractor derived from an episode, which is a different text and may be nothing at all: an episode naming one subject yields a node and no edge. Graphiti searches episodes by BM25 over their content, so retrieval here depends on the stored words rather than extraction. Internal completion-only callers may request identity convergence. Ranked results choose artefact identifiers; every completed episode carrying a selected identifier is then enumerated independently of BM25 so conflicts outside the ranked window remain visible to the caller. Mixed public episode searches may require completion only for Reflection-authored episodes while leaving ordinary historical episodes visible, or require trusted Lens or Reflection writer provenance.

search_episodes(server, group_id, query, max_results)

@spec search_episodes(GenServer.server(), String.t(), String.t(), pos_integer()) ::
  {:ok, [map()]} | {:error, term()}

search_episodes(server, group_id, query, max_results, opts)

@spec search_episodes(
  GenServer.server(),
  String.t(),
  String.t(),
  pos_integer(),
  keyword()
) :: {:ok, [map()]} | {:error, term()}

search_nodes(server \\ __MODULE__, group_id, query, max_results, opts \\ [])

@spec search_nodes(
  GenServer.server(),
  String.t(),
  String.t(),
  pos_integer(),
  keyword()
) ::
  {:ok, [map()]} | {:error, term()}

Node search against group_id via graphiti's search_ with the NODE_HYBRID_SEARCH_RRF recipe. Unlike search/4 (which returns edges and whose node_labels filter matches edges by endpoint), this returns nodes that are not reliably reachable through edge search.

Returns {:ok, [%{name:, summary:, attributes:}]} ordered by relevance.

Options

  • :node_labels — optional [String.t()]. When present, a SearchFilters(node_labels: …) restricts results to nodes carrying one of those labels. When absent, all nodes are eligible.

shared_client_spec(llm_model, embedder_model)

@spec shared_client_spec(map(), map()) :: %{
  llm: map(),
  embedder: map(),
  cross_encoder: map()
}

Decide which provider builds each shared client, from the two configured model specs. Pure — no Pythonx, no credentials read — so the per-role dispatch is pinned deterministically and default_construct_shared_clients/2 is left with nothing to decide.

Each role takes its own spec's provider. The cross-encoder has no spec of its own and follows the llm role. A Google embedder carries :batch_size of 1; the OpenAI embedder takes no batch size at all, its client having no such parameter and no equivalent of the Gemini batching defect.

start_link(opts \\ [])

summarise_python_error(error)

@spec summarise_python_error(Pythonx.Error.t()) :: String.t()

Build a compact one-line reason from a Pythonx.Error.

Exception.message/1 on a Pythonx.Error joins the entire Python traceback (every frame, every embedding vector echoed in a call line) into one multi-line blob. On a transient FalkorDB reset that blob — and the whole embedding search vector inside it — gets dumped to the log via the rescue's {:error, {:python, reason}}.

The struct's :lines field is the output of Python's traceback.format_exception(type, value, traceback): a list whose final non-blank entry is the "ExceptionClass: message" summary line. We take that one line — the error's class and message — and drop the frames entirely.