Contributing

Contributions are welcome! This page covers the development setup, project layout, and how to add new features.

Changesets required

Every PR must include a changeset file generated by running pnpm changeset. See CONTRIBUTING.md for the full guide.

Development setup

pnpm install
pnpm typecheck        # strict TypeScript check
pnpm test             # vitest unit + golden tests
pnpm test:golden      # retrieval recall@k per strategy
pnpm test:bench       # hot-path latency benchmarks
pnpm lint             # eslint
pnpm fmt              # prettier
pnpm build            # tsc + copy migrations + copy dashboard → dist/

Reinstalling the global binary from local build

After making source changes:

pnpm build
npm install -g .      # run from project root

This replaces the globally installed copy with your local dist/.

---

Project layout

src/
  agents/
    claude_code.ts      # Claude Code transcript parser (fallback format)
    codex.ts            # Codex rollout parser + session lookup by id
    index.ts            # adapter registry, format detection
  cli/
    index.ts            # commander CLI entry point
    init.ts             # somtum init — installs hooks + MCP config
    init_codex.ts       # somtum init --agent codex
    serve.ts            # somtum serve — local dashboard server
    stats.ts            # somtum stats
    doctor.ts           # somtum doctor — health checks
    hook.ts             # internal: dispatches hook events by name
    search.ts / show.ts / forget.ts / edit.ts
    list.ts             # somtum list
    reset.ts            # somtum reset — wipe project DB
    export.ts / import.ts / purge.ts / sync.ts / rebuild.ts / reindex.ts
    config_cmd.ts
    suggest_claude_md.ts
  core/
    db.ts               # SQLite setup, migration runner
    store.ts            # MemoryStore — CRUD for observations
    cache.ts            # PromptCache — exact + fuzzy lookup
    retriever/          # bm25, embeddings, hybrid, index, factory
    extractor.ts        # session transcript → observations
    llm_backend.ts      # picks the extraction backend (anthropic / claude-cli / codex-cli)
    index_gen.ts        # renders index.md (incremental past 1k obs)
    memory_files.ts     # writes memories/<YYYY-MM>/<ulid>.md
    retrieval_stats.ts
    embeddings.ts       # Embedder interface + encode/decode utils
    privacy.ts          # redact() — runs on every capture
    tokens.ts           # gpt-tokenizer wrapper
  hooks/
    post_session.ts     # SessionEnd/PreCompact: extract → store → index → warm-start
    pre_prompt.ts       # UserPromptSubmit: cache lookup + auto-inject + false-hit detection
    pre_read.ts         # PreToolUse: file gating
  mcp/                  # MCP server + tool implementations
  dashboard/
    index.html          # single-page dashboard (served by somtum serve)
  config.ts             # global + project config merge
  index.ts              # public API for embedding Somtum
src/db/migrations/      # NNN_name.sql migration files
test/
  golden/               # per-strategy retrieval golden sets
  bench/                # hot-path latency benchmarks
  fixtures/             # synthetic session transcripts

---

Adding a new observation kind

1. Extend the zod enum in src/core/schema.ts
2. Update the extractor prompt in src/core/extractor.ts
3. Add a fixture in test/fixtures/ and an assertion
4. Update src/core/index_gen.ts to render the new section

Adding a new MCP tool

1. Define args + response with zod in src/mcp/tools.ts
2. Register it in src/mcp/server.ts
3. Response must include a tokens field
4. Add an integration test in src/mcp/server.test.ts

Adding a new CLI command

1. Add the command in src/cli/index.ts (commander)
2. Implement the handler in a new file under src/cli/
3. Add a test in test/

Adding a new retrieval strategy

1. Implement the Retriever interface in src/core/retriever/
2. Register it in src/core/retriever/factory.ts
3. Add a golden test set in test/golden/

---

Database migrations

Migrations live in src/db/migrations/ as numbered SQL files (NNN_name.sql). The migration runner in src/core/db.ts applies them in order on startup.

To add a migration:

1. Create src/db/migrations/<next-number>_description.sql
2. Write the SQL (must be idempotent where possible)
3. Test with a fresh DB and an existing DB

---

Release process

Somtum uses changesets for versioning:

pnpm changeset          # describe your change
pnpm changeset version  # bump versions (CI does this)
pnpm release            # publish to npm (CI does this)

---

Adding a new agent

1. Add src/agents/<agent>.ts exporting an AgentAdapter: detect(raw) sniffs the first line of a transcript, parse(raw) returns Turn[], and findTranscript(sessionId) is optional, for agents whose hooks may omit the transcript path.
2. Register it in src/agents/index.ts, ahead of claudeCode, which accepts any input and must stay last.
3. If the agent's hooks use a different payload or reply shape from Claude Code, map it in src/cli/hook.ts. Codex needed no mapping.
4. Add an installer next to src/cli/init_codex.ts and a branch for it in initCommand.
5. Build the parser and its tests from real transcripts, not from documentation, and keep user content out of the fixtures.