diff --git a/01-VISION.md b/01-VISION.md index e6331d9..cd14bd4 100644 --- a/01-VISION.md +++ b/01-VISION.md @@ -35,15 +35,15 @@ Software projects are easier to conceptualize top-down using graphs. Traditional | Trello | Simple, visual | No hierarchy, no agent support | | Kumu / Obsidian Canvas | Graph modeling | Not issue trackers | -**The gap:** No tool combines graph-native project modeling, multi-repo code integration, and AI-agent-first API design. +**The gap:** No tool combines graph-native project modeling, multi-repo code integration, layered information architecture (structure / work / dependencies / context), and AI-agent-first API design. ## Design Principles 1. **Graph is the spine.** The decomposition tree defines project structure. Everything else — views, permissions, agent navigation — derives from graph position. -2. **Two node types.** Components are the skeleton (stable, map to code). Issues are the work (flow through statuses, get assigned). Cleaner than untyped depth-as-type. -3. **Two graphs, separated.** The decomposition tree (strict parent→child) and the association graph (lateral links) are distinct. The tree is structural; links are annotation. -4. **Code is the skeleton.** Connect your repos, infer the component tree. The fastest path from "nothing" to "structured project" is through code you already have. -5. **Agents are first-class actors.** Not assistants bolted on — agents have accounts, roles, permissions, and can traverse the graph independently. +2. **Four layers, one topology.** The graph is organized as four independent overlays on a shared spatial structure — like layers in a GIS. Layer 1 (Structure) is the component skeleton. Layer 2 (Work) is issues and coordination. Layer 3 (Code Connections) is technical dependencies between components. Layer 4 (Artifacts) is external context — docs, designs, media. Each layer answers a different question, has its own update cadence, and can be toggled independently in the UI and agent API. +3. **Three node types.** Components are the skeleton (stable, map to code). Issues are the work (flow through statuses, get assigned). Artifacts are the context (docs, designs, screenshots attached to any node). Cleaner than untyped depth-as-type. +4. **Code is the skeleton.** Connect your repos, infer the component tree. The fastest path from "nothing" to "structured project" is through code you already have. Code analysis also feeds Layer 3 — import graphs and API calls become visible dependency edges. +5. **Agents are first-class actors.** Not assistants bolted on — agents have accounts, roles, permissions, and can traverse the graph independently. Agents can scope their view to specific layers — a triage agent sees only Layer 2, an impact analysis agent sees Layers 1+3. 6. **Granular trust.** The permission system is policy-based from day one. Roles are convenience bundles over a granular engine, not hardcoded ceilings. 7. **Keyboard-first.** Every action has a shortcut. The command palette is the primary navigation method. 8. **Plan-then-apply.** Structural changes show a preview of consequences before committing. diff --git a/02-DATA-MODEL.md b/02-DATA-MODEL.md index a3a823d..5a099dd 100644 --- a/02-DATA-MODEL.md +++ b/02-DATA-MODEL.md @@ -2,16 +2,25 @@ ## Overview -The data model consists of two overlaid graphs on a set of **typed nodes**: +The data model is organized as **four layers** stacked on a shared node topology — like layers in a GIS or Photoshop. Each layer answers a different question, has its own update cadence, and can be toggled independently in the UI and the agent API. -1. **Decomposition Tree** — strict parent→child hierarchy (DAG) -2. **Association Graph** — lateral many-to-many connections with typed edges +| Layer | Name | Question | Update Cadence | +|-------|------|----------|----------------| +| 1 | **Structure** | What is the project made of? | Stable — changes with major refactors | +| 2 | **Work** | What needs to be done? | High churn — changes daily | +| 3 | **Code Connections** | How do parts depend on each other technically? | Changes with the codebase | +| 4 | **Artifacts** | What context exists around this? | Updated as docs/designs evolve | -These are deliberately separated. The decomposition tree is the *spine* of the product. The association graph is an *overlay* that adds context without defining structure. +Two fundamental graph structures span the layers: + +1. **Decomposition Tree** — strict parent→child hierarchy (DAG). The *spine* of the product. Carries Layer 1 (component→component edges) and the attachment of Layer 2 nodes to Layer 1 nodes. +2. **Overlay Edges** — lateral many-to-many connections with typed edges. Each edge type belongs to a specific layer (work coordination links in Layer 2, technical dependency links in Layer 3, artifact attachments in Layer 4). + +The layers share the same spatial structure but carry different information. Layer 1 (Structure) is the base layer — all other layers attach to it. ## Node Types -The graph has two main node types: **components** and **issues**. Components form the skeleton; issues are work attached to that skeleton. +The graph has three node types: **components**, **issues**, and **artifacts**. Components form the skeleton (Layer 1); issues are work attached to that skeleton (Layer 2); artifacts are external context attached to any node (Layer 4). ### Component Node @@ -53,23 +62,48 @@ An issue represents work to be done — a task, bug, feature, spike. Issues atta | `created_by` | actor_id | Creator (user or agent) | | `cycle_id` | cycle_id? | Current cycle membership | +### Artifact Node (Layer 4) + +An artifact represents external context — a document, design file, screenshot, or link — attached to any component or issue. + +| Field | Type | Description | +|-------|------|-------------| +| `id` | UUID | Unique identifier | +| `title` | string | Display name | +| `kind` | enum | `link`, `file`, `embed` | +| `url` | string? | External URL (Figma, Google Docs, Confluence, etc.) | +| `file_ref` | s3_key? | MinIO object key (for uploaded files) | +| `mime_type` | string? | MIME type (for uploaded files) | +| `size_bytes` | int? | File size (for uploaded files) | +| `attached_to` | node_id | Component or issue this artifact belongs to | +| `created_by` | actor_id | Uploader / linker | +| `created_at` | timestamp | Creation time | + +Artifacts do **not** participate in the decomposition tree — they have no parent/child relationships of their own. They attach to components or issues via a dedicated `HAS_ARTIFACT` edge and are purely a Layer 4 overlay. + ### How They Relate ``` -Project (root) -├── Component: auth-service (→ repo: github.com/team/auth) -│ ├── Component: oauth-module (→ directory: /src/oauth) -│ │ ├── Issue: implement refresh tokens -│ │ └── Issue: fix token expiry bug -│ └── Component: session-manager (→ directory: /src/sessions) +Project (root) Layer 1: Structure +├── Component: auth-service (→ repo) ───────────────── +│ ├── Component: oauth-module (→ dir) +│ │ ├── Issue: implement refresh tokens Layer 2: Work +│ │ └── Issue: fix token expiry bug ──────────────── +│ │ └── Artifact: "error-screenshot.png" Layer 4: Artifacts +│ └── Component: session-manager (→ dir) ───────────────── │ └── Issue: add Redis session store -├── Component: frontend-app (→ repo: github.com/team/web) -│ ├── Component: dashboard (→ directory: /src/views/dashboard) -│ └── Component: auth-ui (→ directory: /src/views/auth) +├── Component: frontend-app (→ repo) +│ ├── Component: dashboard (→ dir) +│ │ └── Artifact: "dashboard-figma-mockup" (link) Layer 4 +│ └── Component: auth-ui (→ dir) │ └── Issue: redesign login page +│ +│ [Layer 3 — Code Connections (not shown in tree, these are lateral edges):] +│ auth-service ──IMPORTS──► jwt-lib +│ frontend-app ──CALLS_API──► auth-service ``` -Components are the **skeleton** — stable structure that mirrors your codebase. Issues are the **work** — they flow through statuses, get assigned, join cycles. This separation is cleaner than depth-as-type because a component at depth 3 is still a component, not a "task." +Components are the **skeleton** (Layer 1) — stable structure that mirrors your codebase. Issues are the **work** (Layer 2) — they flow through statuses, get assigned, join cycles. Artifacts are the **context** (Layer 4) — docs, designs, and media that surround the work. Code connections (Layer 3) are lateral edges between components representing technical dependencies. This separation is cleaner than depth-as-type because a component at depth 3 is still a component, not a "task." ### Hierarchy Rules @@ -174,27 +208,50 @@ Moving a node to a different parent is a **structural change** (plan-then-apply) - For issues: moving between components changes which codebase the work is conceptually "in" - System warns about consequences before committing -## Association Graph (Lateral Links) +## Overlay Edges (Lateral Links) -### Link Types +Lateral links are typed edges that do not belong to the decomposition tree. Each link type belongs to a specific layer. -| Type | Semantics | Directionality | -|------|-----------|----------------| -| `blocks` | A blocks B | Directed | -| `blocked_by` | Inverse of blocks | Directed (auto-created) | -| `relates_to` | General association | Undirected | -| `duplicates` | A is a duplicate of B | Directed | -| `depends_on` | Component A depends on component B | Directed | +### Layer 2: Work Coordination Links -`depends_on` is specifically for inter-component dependencies — can be inferred from code (import graphs, API calls) or declared manually. +These connect issues to other issues and describe work-level relationships. + +| Type | Semantics | Directionality | Between | +|------|-----------|----------------|---------| +| `blocks` | A blocks B | Directed | Issue → Issue | +| `blocked_by` | Inverse of blocks | Directed (auto-created) | Issue → Issue | +| `duplicates` | A is a duplicate of B | Directed | Issue → Issue | +| `relates_to` | General work association | Undirected | Issue ↔ Issue | + +### Layer 3: Code Connection Links + +These connect components to other components and describe technical dependencies. They can be auto-inferred from code analysis (import graphs, API routes, shared infrastructure) or declared manually. + +| Type | Semantics | Directionality | Between | +|------|-----------|----------------|---------| +| `depends_on` | A depends on B at build/runtime | Directed | Component → Component | +| `imports` | A imports code from B | Directed | Component → Component | +| `calls_api` | A calls B's API at runtime | Directed | Component → Component | +| `shares_db` | A and B share a database | Undirected | Component ↔ Component | + +Auto-inference sources for Layer 3 links: +- **Import graphs:** static analysis of `import`, `require`, `use` statements across repo boundaries +- **API route matching:** outbound HTTP calls matched to known service endpoints +- **Shared infrastructure:** components linked to the same database, queue, or cache + +Auto-inferred links are tagged `source: inferred` and can be overridden or dismissed manually. + +### Layer 4: Artifact Attachment Links + +These connect artifacts to the nodes they provide context for. Represented by the `HAS_ARTIFACT` edge in the decomposition tree (see Artifact Node above). ### Backlinks (Inbound Link Surfacing) -Every node surfaces inbound lateral links: "3 tasks blocked by this," "2 components depend on this." +Every node surfaces inbound lateral links from all layers: "3 tasks blocked by this" (Layer 2), "2 components depend on this" (Layer 3), "1 design attached" (Layer 4). Backlink counts are shown per-layer when layer toggles are active. ### Cross-Project Links (v0.2+) -Deferred. Will enable cross-project/cross-repo dependency tracking. +Deferred. Will enable cross-project/cross-repo dependency tracking across all layers. ## Triage Inbox diff --git a/03-VIEWS-AND-UI.md b/03-VIEWS-AND-UI.md index 5b08f17..7ffd75b 100644 --- a/03-VIEWS-AND-UI.md +++ b/03-VIEWS-AND-UI.md @@ -4,6 +4,48 @@ Non-Linear provides two primary graph views, a flat fallback, and a cycle view. The interaction model is keyboard-first with a command palette as the primary navigation method. +## Layer Visibility + +All graph views share a **layer toggle** — a set of controls that determine which of the four data layers are visible. Layers are independent overlays on the same spatial structure; toggling a layer on or off does not change node positions, only what information is rendered. + +### Toggle Controls + +A compact toggle bar sits in the view toolbar (near filtering and grouping controls). Each layer is a pill-style toggle: + +``` +[ Structure ] [ Work ] [ Connections ] [ Artifacts ] + ● ● ○ ○ +``` + +Active layers are highlighted; inactive layers are dimmed. At least Layer 1 (Structure) must be active — it cannot be toggled off since it is the base topology. + +### Presets + +Named presets configure common layer combinations with a single click: + +| Preset | Layers Active | Use Case | +|--------|---------------|----------| +| **Default** | Structure + Work | Day-to-day issue tracking — the current behavior | +| **Architecture** | Structure + Connections | Impact analysis, dependency review, onboarding to codebase topology | +| **Context** | Structure + Artifacts | Onboarding, design review, gathering background on a component | +| **Full** | All four layers | Complete picture — useful for deep dives on a specific subtree | + +Presets are accessible via the toggle bar dropdown and via the command palette (`Cmd+K` → "Switch layer preset"). + +### Keyboard Shortcuts + +- `Alt+1` through `Alt+4` — toggle individual layers +- `Alt+0` — cycle through presets + +### Layer Rendering + +When multiple layers are active, each layer contributes distinct visual elements: + +- **Layer 1 (Structure):** Component nodes and parent→child edges. Always the spatial backbone. +- **Layer 2 (Work):** Issue nodes nested under components. Work coordination edges (`blocks`, `duplicates`) drawn as dashed overlays between issues. +- **Layer 3 (Connections):** Technical dependency edges between components. Drawn as colored directional arrows (distinct color per link type: `imports`, `calls_api`, `depends_on`, `shares_db`). +- **Layer 4 (Artifacts):** Small attachment badges on nodes that have artifacts. Clicking a badge opens the artifact list in the detail panel. + ## View 1: Layered Overview **Purpose:** See the entire project structure at a glance, organized by abstraction level. @@ -22,13 +64,14 @@ Layer 3: [OAuth flow] [Bar chart component] ### Behavior -- **Edges visible:** Parent→child edges drawn between layers. Lateral links shown as overlay edges (dashed, colored by type). -- **Backlink count badges:** Nodes show inbound link counts ("3 blocked"). -- **Cycle highlighting:** Cycle members highlighted with color/tag. -- **Collapse/expand:** Each node can collapse its subtree. -- **Zoom:** At high zoom, nodes show only title. At lower zoom, nodes show title + status + assignee. -- **Filtering:** Filter by label, status, assignee. Non-matching nodes dimmed, not hidden (preserving structural context). -- **Node coloring:** Status-based by default. Optionally by label, assignee, or priority. +- **Edges visible:** Parent→child edges always drawn. Overlay edges from active layers rendered on top — Layer 2 work links as dashed lines, Layer 3 code connections as colored directional arrows. +- **Backlink count badges:** Nodes show inbound link counts from active layers ("3 blocked" from Layer 2, "2 depend on this" from Layer 3). +- **Artifact badges:** When Layer 4 is active, nodes with artifacts show a paperclip badge with count. +- **Cycle highlighting:** Cycle members highlighted with color/tag (Layer 2). +- **Collapse/expand:** Each node can collapse its subtree. Collapsing a component hides its children across all active layers. +- **Zoom:** At high zoom, nodes show only title. At lower zoom, nodes show title + status + assignee. Layer 3 edge labels (link type) appear at medium zoom. +- **Filtering:** Filter by label, status, assignee. Non-matching nodes dimmed, not hidden (preserving structural context). Filters interact with layer toggles — filtering by status only affects Layer 2 nodes. +- **Node coloring:** Status-based by default (Layer 2). In Architecture preset (Layer 3 active, Layer 2 off), components colored by dependency fan-out (how many things depend on them). ## View 2: Focus Widget @@ -41,8 +84,10 @@ Layer 3: [OAuth flow] [Bar chart component] - **Breadcrumb navigation:** Full root→node path as clickable segments at the top (e.g., `Project > Backend > Auth > Login Flow > Password Reset`). - **Navigation:** Click any node to make it the new focus ("walking the tree"). - **Detail panel:** Full description, comments, change history timeline (compact, filterable by event type). -- **Backlinks section:** Inbound lateral links grouped by type. -- **Quick actions:** Change status, assign, add labels, add child, add link. +- **Backlinks section:** Inbound lateral links grouped by layer and type. Layer 2 links (blocks, duplicates) and Layer 3 links (depends_on, imports) are shown in separate collapsible groups. +- **Artifacts section:** When Layer 4 is active, a dedicated panel lists attached artifacts — thumbnails for images, icons for document links, download buttons for files. +- **Code connections section:** When Layer 3 is active, shows inbound/outbound technical dependencies for the focused component with link type and target component name. +- **Quick actions:** Change status, assign, add labels, add child, add link, attach artifact. ## View 3: Flat List / Board (Fallback) diff --git a/05-AGENT-INTEGRATION.md b/05-AGENT-INTEGRATION.md index aa73b3e..68c0e2a 100644 --- a/05-AGENT-INTEGRATION.md +++ b/05-AGENT-INTEGRATION.md @@ -34,6 +34,26 @@ The agent helps human users by analyzing the graph and suggesting actions. Governed by the same policy engine as human users. Agents authenticate with API tokens tied to their actor account. +### Layer Filtering + +All read endpoints accept an optional `layers` query parameter that controls which data layers are included in the response. This mirrors the UI layer toggles and lets agents scope their view to only the information they need. + +``` +?layers=structure,work # Default: structure + issues (Layers 1+2) +?layers=structure,connections # Architecture view: structure + code dependencies (Layers 1+3) +?layers=structure,work,connections # Structure + issues + code dependencies (Layers 1+2+3) +?layers=all # All four layers +``` + +| Layer Value | Layer | What's Included | +|-------------|-------|-----------------| +| `structure` | 1 | Component nodes and component→component edges | +| `work` | 2 | Issue nodes, work coordination links (blocks, duplicates, relates_to) | +| `connections` | 3 | Code connection edges (depends_on, imports, calls_api, shares_db) | +| `artifacts` | 4 | Artifact nodes and attachment edges | + +If `layers` is omitted, the default is `structure,work` (backward-compatible with the pre-layer API). The `structure` layer is always implicitly included — it cannot be excluded since it provides the base topology. + ### Graph Traversal ``` @@ -41,10 +61,12 @@ GET /api/v1/nodes/{id} # Read single node GET /api/v1/nodes/{id}/children # Direct children GET /api/v1/nodes/{id}/subtree # Full subtree (depth-limited) GET /api/v1/nodes/{id}/parent # Parent node -GET /api/v1/nodes/{id}/links # Outbound lateral links -GET /api/v1/nodes/{id}/backlinks # Inbound lateral links +GET /api/v1/nodes/{id}/links # Outbound lateral links (filtered by active layers) +GET /api/v1/nodes/{id}/backlinks # Inbound lateral links (filtered by active layers) GET /api/v1/nodes/{id}/path # Path from root to this node GET /api/v1/nodes/{id}/history # Change history timeline +GET /api/v1/nodes/{id}/artifacts # Attached artifacts (Layer 4) +GET /api/v1/nodes/{id}/connections # Code connections for this component (Layer 3) GET /api/v1/projects/{id}/root # Project root node GET /api/v1/projects/{id}/inbox # Triage inbox (rootless nodes) ``` @@ -56,6 +78,7 @@ GET /api/v1/projects/{id}/nodes?status=in_progress&assignee=agent-123 GET /api/v1/projects/{id}/nodes?label=bug&status=backlog GET /api/v1/projects/{id}/nodes?unblocked=true&status=todo GET /api/v1/projects/{id}/nodes?cycle=current +GET /api/v1/projects/{id}/nodes?layers=structure,connections&type=component ``` ### Write Actions (v0.1) @@ -66,13 +89,21 @@ PATCH /api/v1/nodes/{id}/status # Change status PATCH /api/v1/nodes/{id}/parent # Triage: assign parent to inbox item ``` +### Write Actions (v0.1 — Layer 4) + +``` +POST /api/v1/nodes/{id}/artifacts # Attach an artifact (link or file) +DELETE /api/v1/nodes/{id}/artifacts/{aid} # Remove an artifact +``` + ### Deferred Actions (v0.2+) ``` POST /api/v1/nodes/{id}/children # Create child node -POST /api/v1/nodes/{id}/links # Create lateral link +POST /api/v1/nodes/{id}/links # Create lateral link (Layer 2 or Layer 3) DELETE /api/v1/nodes/{id}/links/{link_id} # Remove lateral link PATCH /api/v1/nodes/{id} # Edit node details +POST /api/v1/nodes/{id}/connections # Declare a code connection (Layer 3, manual) ``` ### Authentication @@ -111,7 +142,7 @@ All list endpoints use **cursor-based pagination**. Responses include a cursor f ### Response Format -All responses include effective permissions on the returned resource: +All responses include effective permissions on the returned resource. Links and backlinks are grouped by layer when multiple layers are requested. ```json { @@ -123,34 +154,50 @@ All responses include effective permissions on the returned resource: "labels": ["frontend", "p1"], "parent_id": "uuid-456", "children_count": 3, + "active_layers": ["structure", "work", "connections"], "backlinks": { - "blocked_by_count": 1, - "relates_to_count": 2 - }, - "links": [ - { - "type": "blocked_by", - "target_id": "uuid-789", - "target_accessible": true, - "target_title": "SSO integration" + "work": { + "blocked_by_count": 1, + "relates_to_count": 2 }, - { - "type": "relates_to", - "target_id": "uuid-000", - "target_accessible": false, - "target_title": null + "connections": { + "depended_on_by_count": 3 } - ] + }, + "links": { + "work": [ + { + "type": "blocked_by", + "target_id": "uuid-789", + "target_accessible": true, + "target_title": "SSO integration" + } + ], + "connections": [ + { + "type": "imports", + "target_id": "uuid-lib", + "target_accessible": true, + "target_title": "jwt-lib", + "source": "inferred" + } + ] + }, + "artifacts_count": 2 }, "permissions": { "can_edit": false, "can_change_status": true, "can_comment": true, - "can_create_child": false + "can_create_child": false, + "can_attach_artifact": true } } ``` +When `layers` excludes a layer, the corresponding key is omitted from `links` and `backlinks`. The `artifacts_count` field is only present when `layers` includes `artifacts`. +``` + ## Agent Design Patterns ### Focused Worker Agent @@ -175,6 +222,17 @@ Loop: 4. Post comment with rationale ``` +### Impact Analysis Agent + +``` +Uses layers: structure + connections (Layer 1 + Layer 3) + 1. Query component with ?layers=structure,connections + 2. Traverse DEPENDS_ON / IMPORTS / CALLS_API edges outward + 3. Build a blast radius: "changing auth-service affects 4 downstream components" + 4. Post comment with dependency diagram + 5. Optionally cross-reference with Layer 2 (work): "3 in-progress issues touch affected components" +``` + ### Decomposition Agent (v0.2+) ``` @@ -215,17 +273,22 @@ POST /api/v1/projects/{id}/webhooks ### Event Catalog -| Event | Trigger | -|-------|---------| -| `node.created` | New node (component or issue) created | -| `node.status_changed` | Issue status transition | -| `node.comment_added` | Comment posted on a node | -| `node.link_created` | Lateral link created | -| `node.link_removed` | Lateral link removed | -| `node.reparented` | Node moved to a different parent | -| `node.cycle_added` | Issue added to a cycle | -| `node.deleted` | Node (and subtree) deleted | -| `repo.commit_associated` | Commit auto-associated with a component | +| Event | Layer | Trigger | +|-------|-------|---------| +| `node.created` | 1/2 | New node (component or issue) created | +| `node.status_changed` | 2 | Issue status transition | +| `node.comment_added` | 2 | Comment posted on a node | +| `node.link_created` | 2/3 | Lateral link created (work or code connection) | +| `node.link_removed` | 2/3 | Lateral link removed | +| `node.reparented` | 1/2 | Node moved to a different parent | +| `node.cycle_added` | 2 | Issue added to a cycle | +| `node.deleted` | 1/2 | Node (and subtree) deleted | +| `artifact.attached` | 4 | Artifact attached to a node | +| `artifact.removed` | 4 | Artifact removed from a node | +| `connection.inferred` | 3 | Code connection auto-detected from repo analysis | +| `repo.commit_associated` | 1 | Commit auto-associated with a component | + +Webhook subscriptions can filter by layer: `"events": ["layer:3"]` subscribes to all Layer 3 events only. ### Payload Format diff --git a/09-TECH-STACK-AND-ARCHITECTURE.md b/09-TECH-STACK-AND-ARCHITECTURE.md index f9226fc..8b6dd7f 100644 --- a/09-TECH-STACK-AND-ARCHITECTURE.md +++ b/09-TECH-STACK-AND-ARCHITECTURE.md @@ -48,27 +48,34 @@ ### Neo4j — Graph Topology -Owns the decomposition tree and lateral links: +Owns the decomposition tree and all overlay edges across the four data layers: +- **Node labels:** `Component` (Layer 1), `Issue` (Layer 2), `Artifact` (Layer 4), `Cycle`, `Project` - Node identity (UUID), short ID - Lightweight properties: status, labels, assignee_id, created_at, updated_at -- Parent → child edges (decomposition tree) -- Lateral link edges: blocks, blocked_by, relates_to, duplicates +- **Layer 1 edges:** `HAS_CHILD` between components (decomposition tree) +- **Layer 1→2 edges:** `HAS_CHILD` from components to issues (work attachment) +- **Layer 2 edges:** `BLOCKS`, `DUPLICATES`, `RELATES_TO` between issues (work coordination) +- **Layer 3 edges:** `DEPENDS_ON`, `IMPORTS`, `CALLS_API`, `SHARES_DB` between components (code connections) +- **Layer 4 edges:** `HAS_ARTIFACT` from components/issues to artifacts - Project root references, cycle membership -**Why Neo4j over Postgres recursive CTEs:** Queries like "find all unblocked leaves in this subtree," "critical path through blocks links," "everything 3 hops from this node" are what Cypher is built for. CTEs get painful with lateral links and variable-depth queries. The gap widens in v0.2+ with cross-project edges. +Each edge type is scoped to a single layer, which enables efficient layer-filtered queries — a Cypher query for "show me this subtree with only Layer 3 edges" simply filters by relationship type. + +**Why Neo4j over Postgres recursive CTEs:** Queries like "find all unblocked leaves in this subtree," "critical path through blocks links," "everything 3 hops from this node" are what Cypher is built for. CTEs get painful with lateral links and variable-depth queries. The gap widens with Layer 3 code connections (multi-hop dependency chains) and in v0.2+ with cross-project edges. ### Postgres — Content & Metadata -- **Rich text content:** issue descriptions (markdown) +- **Rich text content:** issue and component descriptions (markdown) - **Comment threads:** body, author, parent_comment_id (threading), timestamps -- **Attachment metadata:** filename, size, mime_type, s3_key, uploader_id, uploaded_at +- **Attachment metadata:** filename, size, mime_type, s3_key, uploader_id, uploaded_at (inline attachments in comments/descriptions) +- **Artifact metadata (Layer 4):** title, kind, url/file_ref, mime_type — rich metadata for external docs, designs, and uploaded files attached to nodes - **User/agent accounts:** profile data, preferences, notification settings - **Project settings:** configuration, member lists, default policies - **Audit logs:** who changed what, when, with before/after snapshots - **Policy definitions:** role templates, custom permission rules -**Linked to Neo4j by UUID.** Neo4j node stores `id: "abc-123"`. Postgres stores full content keyed by same UUID. FastAPI joins them as needed. +**Linked to Neo4j by UUID.** Neo4j node stores `id: "abc-123"`. Postgres stores full content keyed by same UUID. FastAPI joins them as needed. This applies to all node types: Components (Layer 1), Issues (Layer 2), and Artifacts (Layer 4). ### Redis — Caching & Real-Time @@ -104,7 +111,7 @@ Neo4j stores graph topology and lightweight node properties. All content lives i **Node labels and properties:** ```cypher -// Component node +// Layer 1: Component node CREATE (c:Component { id: "uuidv7", short_id: "NL-C12", @@ -121,7 +128,7 @@ CREATE (c:Component { updated_at: datetime() }) -// Issue node +// Layer 2: Issue node CREATE (i:Issue { id: "uuidv7", short_id: "NL-42", @@ -135,6 +142,19 @@ CREATE (i:Issue { updated_at: datetime() }) +// Layer 4: Artifact node +CREATE (a:Artifact { + id: "uuidv7", + title: "Login flow mockup", + kind: "link", // "link" | "file" | "embed" + url: "https://figma.com/...", // for links/embeds + file_ref: null, // MinIO s3_key for uploaded files + mime_type: null, + size_bytes: null, + created_by: "uuidv7", + created_at: datetime() +}) + // Project root (virtual node linking to decomposition tree root) CREATE (p:Project { id: "uuidv7", @@ -143,22 +163,35 @@ CREATE (p:Project { }) ``` -**Relationships:** +**Relationships (organized by layer):** ```cypher -// Decomposition tree (parent → child) -(parent)-[:HAS_CHILD]->(child) +// Decomposition tree (parent → child) — Layer 1 + Layer 2 +(component)-[:HAS_CHILD]->(component) // Layer 1: structure nesting +(component)-[:HAS_CHILD]->(issue) // Layer 1→2: work attached to structure +(issue)-[:HAS_CHILD]->(issue) // Layer 2: sub-tasks -// Lateral links -(a)-[:BLOCKS]->(b) -(a)-[:RELATES_TO]->(b) -(a)-[:DUPLICATES]->(b) -(a)-[:DEPENDS_ON]->(b) // inter-component architectural dependency +// Layer 2: Work coordination links (between issues) +(issue)-[:BLOCKS]->(issue) +(issue)-[:RELATES_TO]->(issue) +(issue)-[:DUPLICATES]->(issue) + +// Layer 3: Code connection links (between components) +(component)-[:DEPENDS_ON {source: "manual"}]->(component) +(component)-[:IMPORTS {source: "inferred"}]->(component) +(component)-[:CALLS_API {source: "inferred"}]->(component) +(component)-[:SHARES_DB {source: "manual"}]->(component) + +// Layer 4: Artifact attachments +(component)-[:HAS_ARTIFACT]->(artifact) +(issue)-[:HAS_ARTIFACT]->(artifact) // Cycle membership (issue)-[:IN_CYCLE]->(cycle:Cycle { id, name, start_date, end_date }) ``` +Layer 3 edges carry a `source` property (`"manual"` or `"inferred"`) to distinguish human-declared dependencies from code-analysis results. + **Indexes:** ```cypher @@ -168,6 +201,7 @@ CREATE INDEX issue_id FOR (i:Issue) ON (i.id); CREATE INDEX issue_short FOR (i:Issue) ON (i.short_id); CREATE INDEX issue_status FOR (i:Issue) ON (i.status); CREATE INDEX issue_assignee FOR (i:Issue) ON (i.assignee_id); +CREATE INDEX artifact_id FOR (a:Artifact) ON (a.id); CREATE INDEX project_id FOR (p:Project) ON (p.id); ``` @@ -199,6 +233,7 @@ class CommentReaction(SQLModel, table=True): created_at: datetime class Attachment(SQLModel, table=True): + """File attached inline to a comment or description (e.g. pasted image).""" id: uuid.UUID = Field(default_factory=uuid7, primary_key=True) node_id: uuid.UUID = Field(foreign_key="nodecontent.id", index=True) filename: str @@ -208,6 +243,20 @@ class Attachment(SQLModel, table=True): uploader_id: uuid.UUID = Field(foreign_key="actor.id") uploaded_at: datetime +class ArtifactContent(SQLModel, table=True): + """Layer 4: external context attached to a component or issue. + Topology (HAS_ARTIFACT edge) lives in Neo4j; rich metadata lives here.""" + id: uuid.UUID = Field(primary_key=True) # matches Neo4j Artifact node id + title: str + kind: str # "link" | "file" | "embed" + url: str | None = None # external URL (Figma, Docs, etc.) + file_ref: str | None = None # MinIO s3_key for uploaded files + mime_type: str | None = None + size_bytes: int | None = None + node_id: uuid.UUID = Field(foreign_key="nodecontent.id", index=True) + created_by: uuid.UUID = Field(foreign_key="actor.id") + created_at: datetime + class Actor(SQLModel, table=True): """Human user or AI agent.""" id: uuid.UUID = Field(default_factory=uuid7, primary_key=True) @@ -322,7 +371,11 @@ non-linear-api/ │ │ ├── models.py # SQLAlchemy/SQLModel models │ │ ├── descriptions.py # Rich text CRUD │ │ ├── comments.py # Comment thread CRUD -│ │ └── attachments.py # Metadata + MinIO upload/download +│ │ ├── attachments.py # Inline attachment metadata + MinIO upload/download +│ │ └── artifacts.py # Layer 4: artifact CRUD (links, files, embeds) +│ ├── connections/ # Layer 3: code connection analysis +│ │ ├── inference.py # Auto-infer dependencies from repo analysis +│ │ └── manual.py # Manual code connection CRUD │ ├── search/ # Meilisearch integration │ │ ├── indexer.py # Index updates on mutations │ │ └── search.py # Query interface @@ -332,13 +385,16 @@ non-linear-api/ │ ├── tasks/ # Taskiq background jobs │ │ ├── webhooks.py # Deliver webhooks to agent endpoints │ │ ├── indexing.py # Async search index updates -│ │ └── notifications.py # Notification delivery +│ │ ├── notifications.py # Notification delivery +│ │ └── connections.py # Layer 3: periodic code connection inference │ └── api/v1/ # Route handlers │ ├── nodes.py # CRUD + tree operations -│ ├── links.py # Lateral link management +│ ├── links.py # Lateral link management (Layer 2 + Layer 3) │ ├── projects.py # Project CRUD │ ├── comments.py # Comment endpoints -│ ├── attachments.py # Upload/download +│ ├── attachments.py # Inline upload/download +│ ├── artifacts.py # Layer 4: artifact endpoints +│ ├── connections.py # Layer 3: code connection endpoints │ ├── search.py # Search endpoint │ └── agent.py # Agent-specific API surface ├── tests/ @@ -470,16 +526,21 @@ Centrifugo handles both live UI updates and notification delivery over WebSocket ### Events Pushed -| Event | Channel | Payload | -|-------|---------|---------| -| `node.status_changed` | `project:{id}` + `node:{id}` | node_id, old_status, new_status, actor | -| `node.created` | `project:{id}` | node_id, parent_id, type, title, actor | -| `node.deleted` | `project:{id}` + `node:{id}` | node_id, actor | -| `node.reparented` | `project:{id}` + `node:{id}` | node_id, old_parent, new_parent, actor | -| `comment.added` | `node:{id}` | comment_id, node_id, author, preview | -| `link.changed` | `project:{id}` | source_id, target_id, link_type, action (created/removed) | -| `assignment.changed` | `project:{id}` + `node:{id}` | node_id, old_assignee, new_assignee | -| `notification` | `user:{id}` | notification object | +| Event | Layer | Channel | Payload | +|-------|-------|---------|---------| +| `node.status_changed` | 2 | `project:{id}` + `node:{id}` | node_id, old_status, new_status, actor | +| `node.created` | 1/2 | `project:{id}` | node_id, parent_id, type, title, actor | +| `node.deleted` | 1/2 | `project:{id}` + `node:{id}` | node_id, actor | +| `node.reparented` | 1/2 | `project:{id}` + `node:{id}` | node_id, old_parent, new_parent, actor | +| `comment.added` | 2 | `node:{id}` | comment_id, node_id, author, preview | +| `link.changed` | 2/3 | `project:{id}` | source_id, target_id, link_type, layer, action (created/removed) | +| `assignment.changed` | 2 | `project:{id}` + `node:{id}` | node_id, old_assignee, new_assignee | +| `artifact.attached` | 4 | `project:{id}` + `node:{id}` | artifact_id, node_id, title, kind, actor | +| `artifact.removed` | 4 | `project:{id}` + `node:{id}` | artifact_id, node_id, actor | +| `connection.inferred` | 3 | `project:{id}` | source_id, target_id, link_type, source: "inferred" | +| `notification` | — | `user:{id}` | notification object | + +The `layer` field on `link.changed` events tells the client which layer the change affects, enabling clients to ignore events for inactive layers. ### Backend Publish Flow