Enhance documentation for Non-Linear project: introduce status transitions model, redefine workspace structure, and detail comments & collaboration features. Update pagination and webhook specifications for clarity and completeness.
This commit is contained in:
Cнесарев Максим
parent
610b9a26e2
commit
9c0ecc5934
@ -91,6 +91,20 @@ Default statuses (customizable per project):
|
|||||||
|
|
||||||
Components don't have status — their "health" is derived from their children's statuses (e.g., "3/7 issues done, 1 blocked").
|
Components don't have status — their "health" is derived from their children's statuses (e.g., "3/7 issues done, 1 blocked").
|
||||||
|
|
||||||
|
### Status Transitions (v0.1 Default)
|
||||||
|
|
||||||
|
v0.1 ships with a **permissive default model** — any status can transition to any other status. This avoids blocking agents or humans with rigid workflows before real usage patterns emerge.
|
||||||
|
|
||||||
|
```
|
||||||
|
backlog ⇄ todo ⇄ in_progress ⇄ in_review ⇄ done
|
||||||
|
↕ ↕ ↕ ↕ ↕
|
||||||
|
cancelled
|
||||||
|
```
|
||||||
|
|
||||||
|
All transitions are bidirectional. Reopening (`done` → `in_progress`) and skipping stages (`backlog` → `done`) are allowed. The system logs every transition with actor + timestamp for auditability.
|
||||||
|
|
||||||
|
**v0.2: Custom Transition Graphs.** Projects will be able to define a custom state machine that restricts allowed transitions (e.g., "only owners can reopen `done` issues," "`in_review` can only move to `done` or `in_progress`"). The default permissive model remains available as a preset.
|
||||||
|
|
||||||
### Labels
|
### Labels
|
||||||
|
|
||||||
Labels handle all classification orthogonal to hierarchy and type:
|
Labels handle all classification orthogonal to hierarchy and type:
|
||||||
@ -194,13 +208,33 @@ A cycle is a named set of issue references with a date range. Components don't j
|
|||||||
|
|
||||||
Every mutation is a stored event with actor + timestamp. For components, this includes repo link changes and skeleton inference events.
|
Every mutation is a stored event with actor + timestamp. For components, this includes repo link changes and skeleton inference events.
|
||||||
|
|
||||||
## Project (v0.1)
|
## Workspace
|
||||||
|
|
||||||
|
A workspace is the top-level organizational container. It groups projects, users, and agents under a single identity boundary.
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `id` | UUID | Unique identifier |
|
||||||
|
| `name` | string | Workspace name |
|
||||||
|
| `slug` | string | URL-safe identifier (`my-team`) |
|
||||||
|
| `projects` | project_id[] | Projects in this workspace |
|
||||||
|
| `members` | actor_id[] | Users and agents with workspace access |
|
||||||
|
| `created_at` | timestamp | Creation time |
|
||||||
|
|
||||||
|
- Users and agents belong to workspaces. A user can belong to multiple workspaces.
|
||||||
|
- Authentik maps to workspace-level identity — one Authentik group per workspace.
|
||||||
|
- Policies are per-project, but membership is per-workspace. A workspace member can be granted access to specific projects via project-level roles.
|
||||||
|
- Billing (if ever introduced) attaches to the workspace entity.
|
||||||
|
- v0.1 supports a single workspace per deployment. The entity exists from day one to avoid a painful migration later.
|
||||||
|
|
||||||
|
## Project
|
||||||
|
|
||||||
| Field | Type | Description |
|
| Field | Type | Description |
|
||||||
|-------|------|-------------|
|
|-------|------|-------------|
|
||||||
| `id` | UUID | Unique identifier |
|
| `id` | UUID | Unique identifier |
|
||||||
| `name` | string | Project name |
|
| `name` | string | Project name |
|
||||||
|
| `workspace_id` | workspace_id | Parent workspace |
|
||||||
| `root_id` | node_id | Root node of decomposition tree |
|
| `root_id` | node_id | Root node of decomposition tree |
|
||||||
| `repos` | repo_ref[] | Connected repositories |
|
| `repos` | repo_ref[] | Connected repositories |
|
||||||
| `created_at` | timestamp | Creation time |
|
| `created_at` | timestamp | Creation time |
|
||||||
| `members` | actor_id[] | Users and agents with access |
|
| `members` | actor_id[] | Users and agents with project-level access |
|
||||||
|
|||||||
@ -61,6 +61,35 @@ Layer 3: [OAuth flow] [Bar chart component]
|
|||||||
|
|
||||||
**Layout:** Flat list of current cycle members, grouped by subtree or status.
|
**Layout:** Flat list of current cycle members, grouped by subtree or status.
|
||||||
|
|
||||||
|
## Comments & Collaboration
|
||||||
|
|
||||||
|
### Threading Model
|
||||||
|
|
||||||
|
Comments are **flat and parented by nodes** — not nested under other comments. Each node has a single comment stream displayed in chronological order. This keeps the model simple and avoids deeply nested threads that become hard to follow.
|
||||||
|
|
||||||
|
Multiple node comment streams can be merged into a single chronological view (e.g., "show all comments across this subtree") for standup-style review.
|
||||||
|
|
||||||
|
### Markdown & Rich References
|
||||||
|
|
||||||
|
Comments use **GitHub-style markdown** with extensions for project references:
|
||||||
|
|
||||||
|
- `@alice`, `@agent-1` — mention a user or agent (triggers notification)
|
||||||
|
- `#NL-42` — link to an issue node
|
||||||
|
- `#NL-C12` — link to a component node
|
||||||
|
- `!123` — link to a merge request / pull request
|
||||||
|
- Bare commit SHAs (7+ chars) — auto-linked to the relevant repo
|
||||||
|
- Standard markdown: headings, lists, code blocks, inline code, bold/italic, images, links
|
||||||
|
|
||||||
|
Markdown is sanitized server-side before storage to prevent XSS (see Security in [09-TECH-STACK-AND-ARCHITECTURE.md](09-TECH-STACK-AND-ARCHITECTURE.md)).
|
||||||
|
|
||||||
|
### Reactions
|
||||||
|
|
||||||
|
Emoji reactions on comments — lightweight feedback without noise. Standard set (thumbs up/down, eyes, rocket, heart, etc.) plus custom emoji support deferred.
|
||||||
|
|
||||||
|
### Agent Comments
|
||||||
|
|
||||||
|
Agents post comments via the same API as humans. For structured output (test results, decomposition proposals, status reports), agents use markdown code blocks or tables within the comment body. No separate "bot message" type — agents are first-class actors in the comment stream, distinguished by their actor identity in the UI.
|
||||||
|
|
||||||
## Interaction Model
|
## Interaction Model
|
||||||
|
|
||||||
### Keyboard-First
|
### Keyboard-First
|
||||||
@ -104,12 +133,26 @@ Users can switch between views freely. The selected node context is preserved
|
|||||||
|
|
||||||
## Notifications (Scoped Watching)
|
## Notifications (Scoped Watching)
|
||||||
|
|
||||||
|
### Watch Scoping
|
||||||
|
|
||||||
- **Watch granularity:** single node, subtree, or project-wide
|
- **Watch granularity:** single node, subtree, or project-wide
|
||||||
- **Event type filters:** status changes, comments, new children, link changes, agent actions
|
- **Event type filters:** status changes, comments, new children, link changes, agent actions
|
||||||
- **Defaults:** watch nodes you created or are assigned to
|
- **Defaults:** watch nodes you created or are assigned to
|
||||||
- **Mute:** temporarily silence a noisy subtree
|
- **Mute:** temporarily silence a noisy subtree
|
||||||
- **Agent filter:** "only notify me when agent-1 changes something"
|
- **Agent filter:** "only notify me when agent-1 changes something"
|
||||||
|
|
||||||
|
### Delivery Channels
|
||||||
|
|
||||||
|
Notifications are delivered through three channels, configurable per user:
|
||||||
|
|
||||||
|
1. **In-app real-time (Centrifugo).** Connected clients receive instant push via Centrifugo WebSocket channels. Notifications appear in an in-app notification center (badge count + dropdown). This is the primary channel.
|
||||||
|
2. **Push notifications.** Web Push API for browser notifications when the tab is backgrounded. Tauri desktop uses native OS notifications via the Tauri notification plugin.
|
||||||
|
3. **Email.** For offline/away users. Two modes per user preference:
|
||||||
|
- **Immediate:** one email per event (for high-priority watches only)
|
||||||
|
- **Digest:** batched summary (hourly or daily, configurable)
|
||||||
|
|
||||||
|
Delivery is handled by the `tasks/notifications.py` Taskiq worker. The worker reads the user's watch configuration + delivery preferences, then fans out to the appropriate channels.
|
||||||
|
|
||||||
## Entry Experience
|
## Entry Experience
|
||||||
|
|
||||||
### New Project (Clean Start)
|
### New Project (Clean Start)
|
||||||
|
|||||||
@ -82,6 +82,33 @@ PATCH /api/v1/nodes/{id} # Edit node details
|
|||||||
- The actor has roles/policies assigned via the policy engine
|
- The actor has roles/policies assigned via the policy engine
|
||||||
- Token scoping: a token can optionally be restricted to a subset of the agent's permissions
|
- Token scoping: a token can optionally be restricted to a subset of the agent's permissions
|
||||||
|
|
||||||
|
### Pagination
|
||||||
|
|
||||||
|
All list endpoints use **cursor-based pagination**. Responses include a cursor for the next page and a flag indicating whether more results exist.
|
||||||
|
|
||||||
|
**Response envelope:**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"items": [ ... ],
|
||||||
|
"next_cursor": "eyJpZCI6IjEyMyIsInRzIjoiMjAyNS0wMS0wMVQwMDowMDowMFoifQ==",
|
||||||
|
"has_more": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Query parameters:**
|
||||||
|
|
||||||
|
| Parameter | Default | Max | Description |
|
||||||
|
|-----------|---------|-----|-------------|
|
||||||
|
| `limit` | 50 | 200 | Items per page |
|
||||||
|
| `cursor` | null | — | Opaque cursor from previous response |
|
||||||
|
| `max_depth` | 3 | 10 | Subtree query depth limit (subtree endpoint only) |
|
||||||
|
|
||||||
|
- **Subtree queries** (`GET /nodes/{id}/subtree`) accept `max_depth` to cap traversal depth. The graph visualization uses this to load progressively — fetch immediate children on expand, not the entire tree at once.
|
||||||
|
- **Children queries** (`GET /nodes/{id}/children`) are paginated but not depth-limited (they return a single level).
|
||||||
|
- **Search/filter queries** (`GET /projects/{id}/nodes?...`) are always paginated.
|
||||||
|
- Cursor encoding is opaque to the client. Internally it encodes the sort key (e.g., `created_at` + `id`) for stable pagination under concurrent writes.
|
||||||
|
|
||||||
### Response Format
|
### Response Format
|
||||||
|
|
||||||
All responses include effective permissions on the returned resource:
|
All responses include effective permissions on the returned resource:
|
||||||
@ -160,14 +187,88 @@ Triggered when: a node has label "needs-decomposition"
|
|||||||
|
|
||||||
## Webhooks (v0.1)
|
## Webhooks (v0.1)
|
||||||
|
|
||||||
Events emitted:
|
Webhooks follow a **GitLab-style hooks model** — per-project registration with event filtering, signed payloads, and automatic retry.
|
||||||
|
|
||||||
- `node.status_changed`
|
### Registration
|
||||||
- `node.comment_added`
|
|
||||||
- `node.created`
|
Webhooks are registered per project via the API or settings UI:
|
||||||
- `node.link_created`
|
|
||||||
- `node.reparented`
|
```
|
||||||
- `node.cycle_added`
|
POST /api/v1/projects/{id}/webhooks
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"url": "https://agent.example.com/hooks/nonlinear",
|
||||||
|
"secret": "whsec_...",
|
||||||
|
"events": ["node.status_changed", "node.comment_added"],
|
||||||
|
"active": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `url` | string | HTTPS endpoint to receive POST requests |
|
||||||
|
| `secret` | string | Shared secret for HMAC-SHA256 signature |
|
||||||
|
| `events` | string[] | Event types to subscribe to (empty = all) |
|
||||||
|
| `active` | bool | Enable/disable without deleting |
|
||||||
|
|
||||||
|
### Event Catalog
|
||||||
|
|
||||||
|
| Event | Trigger |
|
||||||
|
|-------|---------|
|
||||||
|
| `node.created` | New node (component or issue) created |
|
||||||
|
| `node.status_changed` | Issue status transition |
|
||||||
|
| `node.comment_added` | Comment posted on a node |
|
||||||
|
| `node.link_created` | Lateral link created |
|
||||||
|
| `node.link_removed` | Lateral link removed |
|
||||||
|
| `node.reparented` | Node moved to a different parent |
|
||||||
|
| `node.cycle_added` | Issue added to a cycle |
|
||||||
|
| `node.deleted` | Node (and subtree) deleted |
|
||||||
|
| `repo.commit_associated` | Commit auto-associated with a component |
|
||||||
|
|
||||||
|
### Payload Format
|
||||||
|
|
||||||
|
All webhook payloads share a common envelope:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"event": "node.status_changed",
|
||||||
|
"timestamp": "2025-06-15T14:30:00Z",
|
||||||
|
"project_id": "uuid-project",
|
||||||
|
"actor": {
|
||||||
|
"id": "uuid-actor",
|
||||||
|
"type": "user",
|
||||||
|
"name": "alice"
|
||||||
|
},
|
||||||
|
"data": {
|
||||||
|
"node_id": "uuid-123",
|
||||||
|
"short_id": "NL-42",
|
||||||
|
"previous_status": "in_progress",
|
||||||
|
"new_status": "in_review"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `data` field varies by event type. Each event includes the affected node's `node_id` and `short_id` at minimum.
|
||||||
|
|
||||||
|
### Signature Verification
|
||||||
|
|
||||||
|
Every webhook request includes an `X-NonLinear-Signature` header containing an HMAC-SHA256 hex digest of the raw request body, computed with the webhook's shared secret:
|
||||||
|
|
||||||
|
```
|
||||||
|
X-NonLinear-Signature: sha256=a1b2c3d4e5f6...
|
||||||
|
```
|
||||||
|
|
||||||
|
Receivers should verify the signature before processing to confirm authenticity.
|
||||||
|
|
||||||
|
### Delivery & Retry
|
||||||
|
|
||||||
|
- Webhooks are delivered asynchronously via the Taskiq worker.
|
||||||
|
- **Timeout:** 10 seconds per delivery attempt.
|
||||||
|
- **Retry:** 3 attempts with exponential backoff (10s, 60s, 300s).
|
||||||
|
- **Failure handling:** After 3 failed attempts, the event is logged and dropped. After 50 consecutive failures across any events, the webhook is automatically deactivated and the project owner is notified.
|
||||||
|
- **Ordering:** Best-effort chronological. Not guaranteed under high concurrency.
|
||||||
|
|
||||||
## MCP Compatibility (v0.2+)
|
## MCP Compatibility (v0.2+)
|
||||||
|
|
||||||
|
|||||||
@ -10,10 +10,10 @@
|
|||||||
│ Command Palette: vue-command-palette / custom │
|
│ Command Palette: vue-command-palette / custom │
|
||||||
│ Keybindings: VueUse useMagicKeys │
|
│ Keybindings: VueUse useMagicKeys │
|
||||||
│ Icons: Lucide │ Font: Inter │ Motion: @vueuse/motion│
|
│ Icons: Lucide │ Font: Inter │ Motion: @vueuse/motion│
|
||||||
│ State: Pinia │ HTTP: ofetch │ WS: native/socket.io │
|
│ State: Pinia │ HTTP: ofetch │ WS: centrifuge-js │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ CROSS-PLATFORM │
|
│ CROSS-PLATFORM │
|
||||||
│ Desktop: Tauri (wraps Vue app) — v0.1 │
|
│ Desktop: Tauri (thin wrapper, no offline) — v0.1 │
|
||||||
│ Mobile: Capacitor (responsive web first) — v0.2+ │
|
│ Mobile: Capacitor (responsive web first) — v0.2+ │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ BACKEND │
|
│ BACKEND │
|
||||||
@ -21,18 +21,26 @@
|
|||||||
│ Taskiq (async task queue — webhooks, imports, agents) │
|
│ Taskiq (async task queue — webhooks, imports, agents) │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ DATA LAYER │
|
│ DATA LAYER │
|
||||||
│ Neo4j — issue graph (nodes, edges, status, labels) │
|
│ Neo4j — graph topology (nodes, edges, status, labels) │
|
||||||
│ Postgres — content & metadata (rich text, comments, │
|
│ Postgres — content & metadata (rich text, comments, │
|
||||||
│ attachments meta, audit logs, project cfg) │
|
│ attachments meta, audit logs, project cfg) │
|
||||||
│ Redis — caching, WebSocket pub/sub, rate limiting │
|
│ Redis — caching, rate limiting │
|
||||||
│ Meilisearch — full-text search (issues, comments) │
|
│ Meilisearch — full-text search (issues, comments) │
|
||||||
│ MinIO — S3-compatible file storage (attachments) │
|
│ MinIO — S3-compatible file storage (attachments) │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
|
│ REAL-TIME │
|
||||||
|
│ Centrifugo — WebSocket server, live updates, push │
|
||||||
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ AUTH │
|
│ AUTH │
|
||||||
│ Authentik — OIDC, API tokens, role mgmt, SSO-ready │
|
│ Authentik — OIDC, API tokens, role mgmt, SSO-ready │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
|
│ INFRA/OPS │
|
||||||
|
│ Caddy (reverse proxy + TLS) │ Vault (secrets) │
|
||||||
|
│ Prometheus + Grafana (metrics + dashboards) │
|
||||||
|
│ Loki (logs) │ Tempo (traces) │ OpenTelemetry (SDK) │
|
||||||
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ DEPLOYMENT │
|
│ DEPLOYMENT │
|
||||||
│ Docker Compose (dev + self-hosted) │
|
│ Docker Compose (dev + single-node production) │
|
||||||
└─────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -83,6 +91,214 @@ Owns the decomposition tree and lateral links:
|
|||||||
- Postgres stores metadata and S3 key; MinIO stores bytes
|
- Postgres stores metadata and S3 key; MinIO stores bytes
|
||||||
- Migration path to AWS S3: zero code changes
|
- Migration path to AWS S3: zero code changes
|
||||||
|
|
||||||
|
## Concrete Database Schemas
|
||||||
|
|
||||||
|
### UUID Strategy
|
||||||
|
|
||||||
|
All entities use UUIDv7 (time-sortable). Generated application-side by FastAPI before writing to either database. The same UUID is used as the primary key in both Neo4j and Postgres, serving as the cross-database join key.
|
||||||
|
|
||||||
|
### Neo4j Schema
|
||||||
|
|
||||||
|
Neo4j stores graph topology and lightweight node properties. All content lives in Postgres.
|
||||||
|
|
||||||
|
**Node labels and properties:**
|
||||||
|
|
||||||
|
```cypher
|
||||||
|
// Component node
|
||||||
|
CREATE (c:Component {
|
||||||
|
id: "uuidv7",
|
||||||
|
short_id: "NL-C12",
|
||||||
|
title: "auth-service",
|
||||||
|
status: null, // components have no status
|
||||||
|
labels: ["backend", "core"],
|
||||||
|
owner_id: "uuidv7",
|
||||||
|
assignee_id: null,
|
||||||
|
repo_provider: "github",
|
||||||
|
repo_url: "https://github.com/team/auth",
|
||||||
|
repo_path: "/src/oauth",
|
||||||
|
repo_branch: "main",
|
||||||
|
created_at: datetime(),
|
||||||
|
updated_at: datetime()
|
||||||
|
})
|
||||||
|
|
||||||
|
// Issue node
|
||||||
|
CREATE (i:Issue {
|
||||||
|
id: "uuidv7",
|
||||||
|
short_id: "NL-42",
|
||||||
|
title: "implement refresh tokens",
|
||||||
|
status: "todo",
|
||||||
|
labels: ["feature", "p1"],
|
||||||
|
assignee_id: "uuidv7",
|
||||||
|
created_by: "uuidv7",
|
||||||
|
cycle_id: "uuidv7",
|
||||||
|
created_at: datetime(),
|
||||||
|
updated_at: datetime()
|
||||||
|
})
|
||||||
|
|
||||||
|
// Project root (virtual node linking to decomposition tree root)
|
||||||
|
CREATE (p:Project {
|
||||||
|
id: "uuidv7",
|
||||||
|
workspace_id: "uuidv7",
|
||||||
|
root_id: "uuidv7"
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Relationships:**
|
||||||
|
|
||||||
|
```cypher
|
||||||
|
// Decomposition tree (parent → child)
|
||||||
|
(parent)-[:HAS_CHILD]->(child)
|
||||||
|
|
||||||
|
// Lateral links
|
||||||
|
(a)-[:BLOCKS]->(b)
|
||||||
|
(a)-[:RELATES_TO]->(b)
|
||||||
|
(a)-[:DUPLICATES]->(b)
|
||||||
|
(a)-[:DEPENDS_ON]->(b) // inter-component architectural dependency
|
||||||
|
|
||||||
|
// Cycle membership
|
||||||
|
(issue)-[:IN_CYCLE]->(cycle:Cycle { id, name, start_date, end_date })
|
||||||
|
```
|
||||||
|
|
||||||
|
**Indexes:**
|
||||||
|
|
||||||
|
```cypher
|
||||||
|
CREATE INDEX comp_id FOR (c:Component) ON (c.id);
|
||||||
|
CREATE INDEX comp_short FOR (c:Component) ON (c.short_id);
|
||||||
|
CREATE INDEX issue_id FOR (i:Issue) ON (i.id);
|
||||||
|
CREATE INDEX issue_short FOR (i:Issue) ON (i.short_id);
|
||||||
|
CREATE INDEX issue_status FOR (i:Issue) ON (i.status);
|
||||||
|
CREATE INDEX issue_assignee FOR (i:Issue) ON (i.assignee_id);
|
||||||
|
CREATE INDEX project_id FOR (p:Project) ON (p.id);
|
||||||
|
```
|
||||||
|
|
||||||
|
### Postgres Schema (SQLModel)
|
||||||
|
|
||||||
|
Postgres stores all content, metadata, and configuration. Managed via Alembic migrations.
|
||||||
|
|
||||||
|
```python
|
||||||
|
class NodeContent(SQLModel, table=True):
|
||||||
|
"""Rich content for both components and issues."""
|
||||||
|
id: uuid.UUID = Field(primary_key=True) # matches Neo4j node id
|
||||||
|
description: str | None = None # markdown
|
||||||
|
description_html: str | None = None # pre-rendered, sanitized HTML
|
||||||
|
|
||||||
|
class Comment(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
node_id: uuid.UUID = Field(foreign_key="nodecontent.id", index=True)
|
||||||
|
author_id: uuid.UUID = Field(foreign_key="actor.id")
|
||||||
|
body: str # markdown
|
||||||
|
body_html: str # pre-rendered, sanitized HTML
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
class CommentReaction(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
comment_id: uuid.UUID = Field(foreign_key="comment.id", index=True)
|
||||||
|
actor_id: uuid.UUID = Field(foreign_key="actor.id")
|
||||||
|
emoji: str # e.g. "+1", "rocket"
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
class Attachment(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
node_id: uuid.UUID = Field(foreign_key="nodecontent.id", index=True)
|
||||||
|
filename: str
|
||||||
|
size_bytes: int
|
||||||
|
mime_type: str
|
||||||
|
s3_key: str # MinIO object key
|
||||||
|
uploader_id: uuid.UUID = Field(foreign_key="actor.id")
|
||||||
|
uploaded_at: datetime
|
||||||
|
|
||||||
|
class Actor(SQLModel, table=True):
|
||||||
|
"""Human user or AI agent."""
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
type: str # "user" | "agent"
|
||||||
|
name: str
|
||||||
|
email: str | None = None
|
||||||
|
authentik_uid: str | None = None # OIDC subject claim
|
||||||
|
preferences: dict = Field(default_factory=dict) # JSON: theme, notifications, etc.
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
class Workspace(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
name: str
|
||||||
|
slug: str = Field(unique=True, index=True)
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
class WorkspaceMember(SQLModel, table=True):
|
||||||
|
workspace_id: uuid.UUID = Field(foreign_key="workspace.id", primary_key=True)
|
||||||
|
actor_id: uuid.UUID = Field(foreign_key="actor.id", primary_key=True)
|
||||||
|
role: str # workspace-level role
|
||||||
|
joined_at: datetime
|
||||||
|
|
||||||
|
class ProjectConfig(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(primary_key=True) # matches Neo4j Project id
|
||||||
|
workspace_id: uuid.UUID = Field(foreign_key="workspace.id", index=True)
|
||||||
|
name: str
|
||||||
|
settings: dict = Field(default_factory=dict) # JSON: custom statuses, defaults
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
class PolicyRule(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
project_id: uuid.UUID = Field(foreign_key="projectconfig.id", index=True)
|
||||||
|
actor_id: uuid.UUID | None = Field(default=None) # null = role-level
|
||||||
|
role_name: str | None = None
|
||||||
|
action: str # e.g. "read_node", "create_child", "*"
|
||||||
|
resource_scope: str # "global" | "subtree:{node_id}" | "node:{node_id}"
|
||||||
|
effect: str # "allow" | "deny"
|
||||||
|
|
||||||
|
class AuditLog(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
project_id: uuid.UUID = Field(foreign_key="projectconfig.id", index=True)
|
||||||
|
actor_id: uuid.UUID = Field(foreign_key="actor.id")
|
||||||
|
action: str # e.g. "status_changed", "reparented"
|
||||||
|
node_id: uuid.UUID | None = None
|
||||||
|
before: dict | None = None # JSON snapshot
|
||||||
|
after: dict | None = None # JSON snapshot
|
||||||
|
created_at: datetime = Field(index=True)
|
||||||
|
|
||||||
|
class WebhookConfig(SQLModel, table=True):
|
||||||
|
id: uuid.UUID = Field(default_factory=uuid7, primary_key=True)
|
||||||
|
project_id: uuid.UUID = Field(foreign_key="projectconfig.id", index=True)
|
||||||
|
url: str
|
||||||
|
secret_hash: str # hashed, never stored plaintext
|
||||||
|
events: list[str] = Field(default_factory=list)
|
||||||
|
active: bool = True
|
||||||
|
consecutive_failures: int = 0
|
||||||
|
created_at: datetime
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dual-Database Consistency
|
||||||
|
|
||||||
|
Neo4j and Postgres are **not replicated** — they own different data, linked by UUID. Both writes happen in the same API request. The consistency strategy for v0.1:
|
||||||
|
|
||||||
|
### Write Order
|
||||||
|
|
||||||
|
1. **Postgres first.** Open a SQLAlchemy transaction. Write content/metadata. Do not commit yet.
|
||||||
|
2. **Neo4j second.** Perform the graph mutation (create node, update properties, create edge).
|
||||||
|
3. **Commit Postgres.** If Postgres commit succeeds, the operation is complete.
|
||||||
|
|
||||||
|
### Failure Handling
|
||||||
|
|
||||||
|
- **Neo4j write fails:** Rollback the Postgres transaction (it hasn't committed). Clean failure, no orphans.
|
||||||
|
- **Postgres commit fails after Neo4j succeeds:** Issue a compensating operation on Neo4j (delete the node/revert the property change). Log the incident for review.
|
||||||
|
- **Partial Neo4j failure (e.g., network timeout with unknown state):** Flag the UUID for reconciliation review.
|
||||||
|
|
||||||
|
### Reconciliation Job
|
||||||
|
|
||||||
|
A periodic background task (Taskiq, runs every 15 minutes) checks for inconsistencies:
|
||||||
|
|
||||||
|
- UUIDs present in Neo4j but missing from Postgres (orphan graph nodes)
|
||||||
|
- UUIDs present in Postgres `NodeContent` but missing from Neo4j (orphan content)
|
||||||
|
- Mismatched lightweight properties (status, assignee) between Neo4j and Postgres audit log
|
||||||
|
|
||||||
|
Orphans are logged and surfaced in an admin dashboard. Auto-repair is deferred — manual review for v0.1.
|
||||||
|
|
||||||
|
### What's Eventually Consistent
|
||||||
|
|
||||||
|
- **Meilisearch index:** Updated asynchronously via Taskiq. Acceptable lag of seconds.
|
||||||
|
- **Redis cache:** Invalidated on mutation. TTL-based expiry as fallback.
|
||||||
|
- **Centrifugo events:** Fire-and-forget publish. Missed events are recoverable by client re-fetch.
|
||||||
|
|
||||||
## Backend Architecture
|
## Backend Architecture
|
||||||
|
|
||||||
### FastAPI Application Structure
|
### FastAPI Application Structure
|
||||||
@ -178,6 +394,56 @@ Not replicated — they own different data. Linked by UUID. Both operations happ
|
|||||||
- **AI agents:** API tokens issued through Authentik, tied to agent actor accounts.
|
- **AI agents:** API tokens issued through Authentik, tied to agent actor accounts.
|
||||||
- **FastAPI:** pure resource server. Validates tokens, reads claims, enforces policies.
|
- **FastAPI:** pure resource server. Validates tokens, reads claims, enforces policies.
|
||||||
|
|
||||||
|
## API Error Contract
|
||||||
|
|
||||||
|
All error responses use a consistent envelope:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"code": "validation_error",
|
||||||
|
"message": "Human-readable description",
|
||||||
|
"details": [
|
||||||
|
{ "field": "title", "message": "Field is required" }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### HTTP Status Codes
|
||||||
|
|
||||||
|
| Code | Usage |
|
||||||
|
|------|-------|
|
||||||
|
| `400` | Malformed request (bad JSON, missing required fields) |
|
||||||
|
| `404` | Resource not found **or** actor lacks permission to see it. Permission-denied nodes return 404 (not 403) to prevent information leakage about resource existence. |
|
||||||
|
| `409` | Conflict (e.g., duplicate `short_id`, stale update) |
|
||||||
|
| `422` | Validation error. Standard FastAPI/Pydantic response with field-level detail. |
|
||||||
|
| `429` | Rate limited. Includes `Retry-After` header (seconds). |
|
||||||
|
| `500` | Internal server error. Logged with correlation ID for debugging. |
|
||||||
|
|
||||||
|
### Rate Limiting
|
||||||
|
|
||||||
|
- Agent API: token bucket per actor, configurable per role (default: 100 req/min).
|
||||||
|
- Human API: higher limits (default: 300 req/min).
|
||||||
|
- Enforced via Redis. `429` response includes `Retry-After` and `X-RateLimit-Remaining` headers.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
### Input Sanitization
|
||||||
|
|
||||||
|
- **Cypher injection:** All Neo4j queries use parameterized Cypher exclusively. User-supplied values are never interpolated into query strings. The `graph/queries.py` module enforces this by accepting only typed parameters.
|
||||||
|
- **SQL injection:** SQLModel/SQLAlchemy parameterized queries. No raw SQL with string formatting.
|
||||||
|
- **XSS prevention:** All markdown content (descriptions, comments) is sanitized server-side using `nh3` (Rust-based HTML sanitizer) before storage. Both raw markdown and pre-rendered sanitized HTML are stored. The frontend renders the pre-sanitized HTML.
|
||||||
|
- **File upload validation:** MIME type validation against allowlist (images, PDFs, common doc formats). Size limit: 25 MB per file. Filename sanitization to prevent path traversal.
|
||||||
|
|
||||||
|
### Transport & Headers
|
||||||
|
|
||||||
|
- **TLS:** All traffic encrypted via Caddy reverse proxy (automatic Let's Encrypt certificates).
|
||||||
|
- **CSRF:** SameSite=Lax cookies for browser sessions. Bearer token API calls are inherently CSRF-safe.
|
||||||
|
- **Content-Security-Policy:** Strict CSP headers served by Caddy. `script-src 'self'`, no inline scripts, no `eval`.
|
||||||
|
- **CORS:** Allowlist of known origins (frontend domain). No wildcard in production.
|
||||||
|
- **Security headers:** `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Strict-Transport-Security`.
|
||||||
|
|
||||||
## Design Language
|
## Design Language
|
||||||
|
|
||||||
Targets Linear's aesthetic: minimal, fast, slightly dark-IDE feel.
|
Targets Linear's aesthetic: minimal, fast, slightly dark-IDE feel.
|
||||||
@ -190,28 +456,196 @@ Targets Linear's aesthetic: minimal, fast, slightly dark-IDE feel.
|
|||||||
- **Animations:** subtle slides and fades, 100-150ms, nothing bouncy
|
- **Animations:** subtle slides and fades, 100-150ms, nothing bouncy
|
||||||
- **Optimistic updates:** every interaction feels instant, syncs in background
|
- **Optimistic updates:** every interaction feels instant, syncs in background
|
||||||
|
|
||||||
## Docker Compose (Dev)
|
## Real-Time Updates (Centrifugo)
|
||||||
|
|
||||||
|
Centrifugo handles both live UI updates and notification delivery over WebSocket. Redis is no longer used for WebSocket pub/sub directly — Centrifugo manages its own connections and subscribes to events published by the backend via its server API.
|
||||||
|
|
||||||
|
### Channel Structure
|
||||||
|
|
||||||
|
| Channel | Scope | Subscribers |
|
||||||
|
|---------|-------|-------------|
|
||||||
|
| `project:{id}` | All mutations in a project | All connected project members |
|
||||||
|
| `node:{id}` | Mutations to a specific node | Clients viewing the focus widget for that node |
|
||||||
|
| `user:{id}` | Personal notifications | Single user's connected clients |
|
||||||
|
|
||||||
|
### Events Pushed
|
||||||
|
|
||||||
|
| Event | Channel | Payload |
|
||||||
|
|-------|---------|---------|
|
||||||
|
| `node.status_changed` | `project:{id}` + `node:{id}` | node_id, old_status, new_status, actor |
|
||||||
|
| `node.created` | `project:{id}` | node_id, parent_id, type, title, actor |
|
||||||
|
| `node.deleted` | `project:{id}` + `node:{id}` | node_id, actor |
|
||||||
|
| `node.reparented` | `project:{id}` + `node:{id}` | node_id, old_parent, new_parent, actor |
|
||||||
|
| `comment.added` | `node:{id}` | comment_id, node_id, author, preview |
|
||||||
|
| `link.changed` | `project:{id}` | source_id, target_id, link_type, action (created/removed) |
|
||||||
|
| `assignment.changed` | `project:{id}` + `node:{id}` | node_id, old_assignee, new_assignee |
|
||||||
|
| `notification` | `user:{id}` | notification object |
|
||||||
|
|
||||||
|
### Backend Publish Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Mutation request → Postgres + Neo4j writes
|
||||||
|
→ Centrifugo server API: publish event to relevant channels
|
||||||
|
→ Taskiq: queue webhook delivery + search index update
|
||||||
|
→ Response to client
|
||||||
|
```
|
||||||
|
|
||||||
|
The backend publishes to Centrifugo via its HTTP server API (not through Redis pub/sub). This gives direct control over which channels receive which events.
|
||||||
|
|
||||||
|
### Client-Side Handling
|
||||||
|
|
||||||
|
- **Pinia store:** Incoming Centrifugo events are applied to the Pinia store. The graph view, focus widget, and list view all react to store changes.
|
||||||
|
- **Optimistic updates:** The client applies mutations locally before the server responds. If the server rejects the mutation (4xx), the client reverts the optimistic change by re-fetching the affected node.
|
||||||
|
- **Conflict model:** Last-write-wins for simple fields (status, assignee, labels). The server is the source of truth. When two clients modify the same field concurrently, the last write committed to Neo4j is the one that Centrifugo broadcasts.
|
||||||
|
- **Reconnection:** On WebSocket disconnect, the client re-subscribes to channels and fetches the current state to catch up on missed events.
|
||||||
|
|
||||||
|
### Cross-Platform
|
||||||
|
|
||||||
|
- **Tauri desktop:** No offline support. Tauri wraps the Vue app as-is. When the network is unavailable, the app shows a connection-lost banner and retries. No local mutation queue.
|
||||||
|
|
||||||
|
## Docker Compose
|
||||||
|
|
||||||
|
### Development
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
api: # FastAPI
|
api: # FastAPI (uvicorn --reload)
|
||||||
frontend: # Vue 3 (vite dev / nginx prod)
|
frontend: # Vue 3 (vite dev server)
|
||||||
worker: # Taskiq worker (same codebase as api)
|
worker: # Taskiq worker (same codebase as api)
|
||||||
neo4j: # Graph database
|
neo4j: # Graph database
|
||||||
postgres: # Relational database
|
postgres: # Relational database
|
||||||
redis: # Cache + pub/sub
|
redis: # Cache + rate limiting
|
||||||
meilisearch: # Search engine
|
meilisearch: # Search engine
|
||||||
minio: # Object storage
|
minio: # Object storage
|
||||||
|
centrifugo: # Real-time WebSocket server
|
||||||
authentik: # Identity provider (server + worker)
|
authentik: # Identity provider (server + worker)
|
||||||
authentik-db: # Authentik's own Postgres
|
authentik-db: # Authentik's own Postgres
|
||||||
```
|
```
|
||||||
|
|
||||||
~10 containers. Runs comfortably on 16GB RAM.
|
~12 containers. Runs comfortably on 16GB RAM.
|
||||||
|
|
||||||
|
### Production (Single-Node)
|
||||||
|
|
||||||
|
Same Docker Compose topology with production-grade additions:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
# ... all of the above, plus:
|
||||||
|
caddy: # Reverse proxy + automatic TLS
|
||||||
|
vault: # Secrets management (HashiCorp Vault)
|
||||||
|
prometheus: # Metrics collection
|
||||||
|
grafana: # Dashboards + alerting
|
||||||
|
loki: # Log aggregation
|
||||||
|
tempo: # Distributed tracing
|
||||||
|
```
|
||||||
|
|
||||||
|
~18 containers total. Recommended: 32GB RAM, 4+ CPU cores for production.
|
||||||
|
|
||||||
|
## Reverse Proxy (Caddy)
|
||||||
|
|
||||||
|
Caddy serves as the single entry point for all traffic:
|
||||||
|
|
||||||
|
- **Automatic TLS** via Let's Encrypt (ACME). Zero-config HTTPS.
|
||||||
|
- **Routes:** `/api/*` → FastAPI, `/ws/*` → Centrifugo, `/*` → Vue frontend (nginx or static files).
|
||||||
|
- **Security headers:** CSP, HSTS, X-Frame-Options, X-Content-Type-Options injected at this layer.
|
||||||
|
- **Rate limiting:** Basic connection-level rate limiting as a first defense layer (application-level rate limiting in FastAPI for finer control).
|
||||||
|
|
||||||
|
## Secrets Management
|
||||||
|
|
||||||
|
### HashiCorp Vault (Primary)
|
||||||
|
|
||||||
|
- All sensitive configuration (database passwords, Authentik client secrets, agent API token signing keys, webhook HMAC secrets, MinIO credentials) stored in Vault.
|
||||||
|
- FastAPI reads secrets from Vault at startup via the `hvac` Python client.
|
||||||
|
- Secret rotation supported without application restart (Vault dynamic secrets for Postgres credentials).
|
||||||
|
|
||||||
|
### Docker Secrets (Fallback)
|
||||||
|
|
||||||
|
For simpler deployments that don't want Vault overhead, Docker secrets via compose files are supported. Environment variables as the last resort.
|
||||||
|
|
||||||
|
## Observability
|
||||||
|
|
||||||
|
### Metrics (Prometheus + Grafana)
|
||||||
|
|
||||||
|
- **FastAPI:** `prometheus-fastapi-instrumentator` exposes request latency, status codes, in-flight requests at `/metrics`.
|
||||||
|
- **Neo4j:** Neo4j Prometheus plugin or `neo4j-exporter` for query latency, cache hit rates, transaction counts.
|
||||||
|
- **Postgres:** `postgres_exporter` for connection pool, query stats, replication lag.
|
||||||
|
- **Redis:** `redis_exporter` for memory, hit rate, connected clients.
|
||||||
|
- **Centrifugo:** Built-in Prometheus metrics for connections, channels, messages.
|
||||||
|
- **Grafana dashboards:** Pre-built dashboards for each service. Alerting rules for error rate spikes, high latency, container restarts.
|
||||||
|
|
||||||
|
### Tracing (OpenTelemetry + Tempo)
|
||||||
|
|
||||||
|
- OpenTelemetry SDK instrumented in FastAPI. Traces span the full request lifecycle: auth → policy check → Neo4j query → Postgres query → response.
|
||||||
|
- Trace context propagated to Taskiq workers (webhook delivery, indexing).
|
||||||
|
- Traces stored in Grafana Tempo, queryable from Grafana.
|
||||||
|
|
||||||
|
### Logging (Structured JSON + Loki)
|
||||||
|
|
||||||
|
- All services emit structured JSON logs (Python `structlog` for FastAPI).
|
||||||
|
- Fields: timestamp, level, correlation_id, actor_id, action, duration_ms.
|
||||||
|
- Collected by Grafana Loki via Docker logging driver or Promtail.
|
||||||
|
- Correlation ID links logs across FastAPI → Taskiq → Centrifugo for a single request.
|
||||||
|
|
||||||
|
### Health Checks
|
||||||
|
|
||||||
|
Every service exposes a health check endpoint used by Docker Compose `healthcheck` directives:
|
||||||
|
|
||||||
|
- `GET /health` on FastAPI, Centrifugo
|
||||||
|
- TCP checks for Neo4j, Postgres, Redis, Meilisearch, MinIO
|
||||||
|
- Grafana alerts on health check failures.
|
||||||
|
|
||||||
|
## Database Migrations
|
||||||
|
|
||||||
|
### Postgres (Alembic)
|
||||||
|
|
||||||
|
- Alembic manages all Postgres schema migrations.
|
||||||
|
- Migration files stored in `alembic/versions/`.
|
||||||
|
- Auto-generated from SQLModel model changes (`alembic revision --autogenerate`).
|
||||||
|
- Applied on deployment: `alembic upgrade head` runs before the API container starts.
|
||||||
|
|
||||||
|
### Neo4j (Versioned Cypher Scripts)
|
||||||
|
|
||||||
|
- Migration scripts stored in `neo4j/migrations/` as numbered Cypher files (`001_initial_schema.cypher`, `002_add_cycle_nodes.cypher`).
|
||||||
|
- A lightweight migration runner (Python script) tracks applied migrations in a Neo4j `:Migration` node.
|
||||||
|
- Applied on deployment before the API container starts.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
### Integration Tests (Primary)
|
||||||
|
|
||||||
|
- **Framework:** pytest with testcontainers.
|
||||||
|
- **Containers:** Neo4j, Postgres, Redis, Meilisearch spun up per test session (shared across tests for speed, reset between test classes).
|
||||||
|
- **Scope:** API endpoint tests hitting real databases. Policy engine tests with real Neo4j graph structures. Dual-DB consistency tests verifying write-order semantics.
|
||||||
|
- **Fixtures:** Factory functions that create graph structures (components, issues, links) for test scenarios.
|
||||||
|
|
||||||
|
### End-to-End Tests
|
||||||
|
|
||||||
|
- **Framework:** Playwright against the full Docker Compose stack.
|
||||||
|
- **Scope:** Critical user flows — create project, add components, navigate graph, triage inbox, agent API workflows.
|
||||||
|
- **Environment:** Dedicated `docker-compose.test.yml` with ephemeral containers.
|
||||||
|
|
||||||
|
### What's Not Mandated
|
||||||
|
|
||||||
|
Isolated unit tests are not required by convention. The dual-DB architecture makes mocking both databases brittle. Integration tests with real containers are the priority.
|
||||||
|
|
||||||
|
## CI/CD Pipeline
|
||||||
|
|
||||||
|
```
|
||||||
|
push/MR → lint → test → build → deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
| Stage | Tools | Description |
|
||||||
|
|-------|-------|-------------|
|
||||||
|
| **Lint** | ruff (Python), eslint + prettier (Vue/TS) | Code style and static analysis |
|
||||||
|
| **Test** | pytest + testcontainers, Playwright | Integration + E2E tests |
|
||||||
|
| **Build** | Docker | Build API, frontend, worker images |
|
||||||
|
| **Push** | Container registry | Push tagged images to GitLab Container Registry |
|
||||||
|
| **Deploy** | SSH + docker compose pull | Pull new images on production server, rolling restart |
|
||||||
|
|
||||||
|
CI runs on GitLab CI. Pipeline definition in `.gitlab-ci.yml`. Testcontainers require Docker-in-Docker or a privileged runner.
|
||||||
|
|
||||||
## Open Technical Questions
|
## Open Technical Questions
|
||||||
|
|
||||||
1. **Graph viz library:** D3 vs Cytoscape — prototype comparison pending
|
1. **Graph viz library:** D3 vs Cytoscape — prototype comparison pending
|
||||||
2. **Real-time strategy:** WebSocket via FastAPI or dedicated service (Centrifugo)?
|
2. **Neo4j driver:** official `neo4j` Python driver vs `neomodel` OGM
|
||||||
3. **Neo4j driver:** official `neo4j` Python driver vs `neomodel` OGM
|
3. **Gantt implementation:** custom or frappe-gantt as starting point
|
||||||
4. **Rich text format:** Markdown (simple) vs ProseMirror JSON (collaborative editing)
|
|
||||||
5. **Gantt implementation:** custom or frappe-gantt as starting point
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user