218 lines
11 KiB
Markdown
218 lines
11 KiB
Markdown
# Non-Linear: Tech Stack & Architecture
|
|
|
|
## Stack Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ FRONTEND │
|
|
│ Vue 3 + Tailwind + Headless UI + ECharts │
|
|
│ Graph Viz: TBD (D3 vs Cytoscape — eval pending) │
|
|
│ Command Palette: vue-command-palette / custom │
|
|
│ Keybindings: VueUse useMagicKeys │
|
|
│ Icons: Lucide │ Font: Inter │ Motion: @vueuse/motion│
|
|
│ State: Pinia │ HTTP: ofetch │ WS: native/socket.io │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ CROSS-PLATFORM │
|
|
│ Desktop: Tauri (wraps Vue app) — v0.1 │
|
|
│ Mobile: Capacitor (responsive web first) — v0.2+ │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ BACKEND │
|
|
│ FastAPI (Python) │
|
|
│ Taskiq (async task queue — webhooks, imports, agents) │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ DATA LAYER │
|
|
│ Neo4j — issue graph (nodes, edges, status, labels) │
|
|
│ Postgres — content & metadata (rich text, comments, │
|
|
│ attachments meta, audit logs, project cfg) │
|
|
│ Redis — caching, WebSocket pub/sub, rate limiting │
|
|
│ Meilisearch — full-text search (issues, comments) │
|
|
│ MinIO — S3-compatible file storage (attachments) │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ AUTH │
|
|
│ Authentik — OIDC, API tokens, role mgmt, SSO-ready │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ DEPLOYMENT │
|
|
│ Docker Compose (dev + self-hosted) │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Data Boundary
|
|
|
|
### Neo4j — Graph Topology
|
|
|
|
Owns the decomposition tree and lateral links:
|
|
|
|
- Node identity (UUID), short ID
|
|
- Lightweight properties: status, labels, assignee_id, created_at, updated_at
|
|
- Parent → child edges (decomposition tree)
|
|
- Lateral link edges: blocks, blocked_by, relates_to, duplicates
|
|
- Project root references, cycle membership
|
|
|
|
**Why Neo4j over Postgres recursive CTEs:** Queries like "find all unblocked leaves in this subtree," "critical path through blocks links," "everything 3 hops from this node" are what Cypher is built for. CTEs get painful with lateral links and variable-depth queries. The gap widens in v0.2+ with cross-project edges.
|
|
|
|
### Postgres — Content & Metadata
|
|
|
|
- **Rich text content:** issue descriptions (markdown)
|
|
- **Comment threads:** body, author, parent_comment_id (threading), timestamps
|
|
- **Attachment metadata:** filename, size, mime_type, s3_key, uploader_id, uploaded_at
|
|
- **User/agent accounts:** profile data, preferences, notification settings
|
|
- **Project settings:** configuration, member lists, default policies
|
|
- **Audit logs:** who changed what, when, with before/after snapshots
|
|
- **Policy definitions:** role templates, custom permission rules
|
|
|
|
**Linked to Neo4j by UUID.** Neo4j node stores `id: "abc-123"`. Postgres stores full content keyed by same UUID. FastAPI joins them as needed.
|
|
|
|
### Redis — Caching & Real-Time
|
|
|
|
- Subtree query cache (TTL, invalidated on graph mutations)
|
|
- WebSocket pub/sub for real-time updates
|
|
- Rate limiting for agent API
|
|
- Authentik token validation cache
|
|
|
|
### Meilisearch — Search Index
|
|
|
|
- Indexes issue titles, descriptions, comments, labels
|
|
- Fed from both Neo4j and Postgres
|
|
- Powers command palette search (issues + commands in one result set)
|
|
- Typo-tolerant, prefix search, filtering by label/status/assignee
|
|
|
|
### MinIO — File Storage
|
|
|
|
- S3-compatible API, self-hosted
|
|
- Stores attachment files (images, docs)
|
|
- Postgres stores metadata and S3 key; MinIO stores bytes
|
|
- Migration path to AWS S3: zero code changes
|
|
|
|
## Backend Architecture
|
|
|
|
### FastAPI Application Structure
|
|
|
|
```
|
|
non-linear-api/
|
|
├── app/
|
|
│ ├── main.py # App, middleware, startup/shutdown
|
|
│ ├── config.py # Settings from env vars
|
|
│ ├── dependencies.py # Shared deps (db sessions, auth, current_user)
|
|
│ ├── auth/ # Authentik integration
|
|
│ │ ├── oidc.py # Token validation, OIDC discovery
|
|
│ │ ├── permissions.py # Policy engine evaluation
|
|
│ │ └── agent_tokens.py # API token management for agents
|
|
│ ├── graph/ # Neo4j layer
|
|
│ │ ├── connection.py # Neo4j driver management
|
|
│ │ ├── queries.py # Cypher query templates
|
|
│ │ ├── mutations.py # Graph write operations
|
|
│ │ └── traversal.py # Subtree, path, neighbor queries
|
|
│ ├── content/ # Postgres layer
|
|
│ │ ├── models.py # SQLAlchemy/SQLModel models
|
|
│ │ ├── descriptions.py # Rich text CRUD
|
|
│ │ ├── comments.py # Comment thread CRUD
|
|
│ │ └── attachments.py # Metadata + MinIO upload/download
|
|
│ ├── search/ # Meilisearch integration
|
|
│ │ ├── indexer.py # Index updates on mutations
|
|
│ │ └── search.py # Query interface
|
|
│ ├── realtime/ # WebSocket layer
|
|
│ │ ├── manager.py # Connection management
|
|
│ │ └── events.py # Event types and broadcasting
|
|
│ ├── tasks/ # Taskiq background jobs
|
|
│ │ ├── webhooks.py # Deliver webhooks to agent endpoints
|
|
│ │ ├── indexing.py # Async search index updates
|
|
│ │ └── notifications.py # Notification delivery
|
|
│ └── api/v1/ # Route handlers
|
|
│ ├── nodes.py # CRUD + tree operations
|
|
│ ├── links.py # Lateral link management
|
|
│ ├── projects.py # Project CRUD
|
|
│ ├── comments.py # Comment endpoints
|
|
│ ├── attachments.py # Upload/download
|
|
│ ├── search.py # Search endpoint
|
|
│ └── agent.py # Agent-specific API surface
|
|
├── tests/
|
|
├── alembic/ # Postgres migrations
|
|
├── docker-compose.yml
|
|
└── pyproject.toml
|
|
```
|
|
|
|
### Request Flows
|
|
|
|
**Typical read ("get node with full context"):**
|
|
|
|
```
|
|
Client → FastAPI → Auth middleware (validate token via Authentik)
|
|
→ Policy engine (check permissions)
|
|
→ Neo4j: fetch node + parent + children + links
|
|
→ Postgres: fetch description, comments, attachment meta
|
|
→ Merge response → Client
|
|
```
|
|
|
|
**Typical write ("change node status"):**
|
|
|
|
```
|
|
Client → FastAPI → Auth → Policy engine
|
|
→ Neo4j: update node status
|
|
→ Redis: invalidate cache, publish event
|
|
→ Taskiq: queue webhook delivery, search index update
|
|
→ WebSocket: broadcast to connected clients
|
|
→ Response → Client
|
|
```
|
|
|
|
### Sync Strategy (Neo4j ↔ Postgres)
|
|
|
|
Not replicated — they own different data. Linked by UUID. Both operations happen in same API request. Compensating transaction pattern for consistency. Eventual consistency acceptable for search index and cache.
|
|
|
|
## Auth Architecture
|
|
|
|
```
|
|
┌──────────┐ OIDC token ┌───────────┐
|
|
│ Vue App ├─────────────────────►│ Authentik │
|
|
└────┬─────┘ (login flow) └─────┬─────┘
|
|
│ │
|
|
│ Bearer token │ Token introspection
|
|
▼ ▼
|
|
┌──────────┐◄────────────────────┌───────────┐
|
|
│ FastAPI │ validate token │ Authentik │
|
|
│ (resource│ check claims │ (OIDC │
|
|
│ server) │ │ provider)│
|
|
└──────────┘ └───────────┘
|
|
```
|
|
|
|
- **Human users:** OIDC login flow. JWT access tokens.
|
|
- **AI agents:** API tokens issued through Authentik, tied to agent actor accounts.
|
|
- **FastAPI:** pure resource server. Validates tokens, reads claims, enforces policies.
|
|
|
|
## Design Language
|
|
|
|
Targets Linear's aesthetic: minimal, fast, slightly dark-IDE feel.
|
|
|
|
- **Spacing:** tight, no wasted space
|
|
- **Colors:** muted base palette, high-contrast accents only for status/priority
|
|
- **Borders:** almost none — separation via spacing and subtle background shifts
|
|
- **Dark mode:** default, light mode secondary
|
|
- **Typography:** Inter, small-but-readable sizes
|
|
- **Animations:** subtle slides and fades, 100-150ms, nothing bouncy
|
|
- **Optimistic updates:** every interaction feels instant, syncs in background
|
|
|
|
## Docker Compose (Dev)
|
|
|
|
```yaml
|
|
services:
|
|
api: # FastAPI
|
|
frontend: # Vue 3 (vite dev / nginx prod)
|
|
worker: # Taskiq worker (same codebase as api)
|
|
neo4j: # Graph database
|
|
postgres: # Relational database
|
|
redis: # Cache + pub/sub
|
|
meilisearch: # Search engine
|
|
minio: # Object storage
|
|
authentik: # Identity provider (server + worker)
|
|
authentik-db: # Authentik's own Postgres
|
|
```
|
|
|
|
~10 containers. Runs comfortably on 16GB RAM.
|
|
|
|
## Open Technical Questions
|
|
|
|
1. **Graph viz library:** D3 vs Cytoscape — prototype comparison pending
|
|
2. **Real-time strategy:** WebSocket via FastAPI or dedicated service (Centrifugo)?
|
|
3. **Neo4j driver:** official `neo4j` Python driver vs `neomodel` OGM
|
|
4. **Rich text format:** Markdown (simple) vs ProseMirror JSON (collaborative editing)
|
|
5. **Gantt implementation:** custom or frappe-gantt as starting point
|