feat: TanStack Query Migration Phase 3 - Knowledge Base Feature

[Wirasm]

Summary

Phase 3 of the TanStack Query migration - migrating the Knowledge Base feature to the modern /features architecture.

Changes

This PR will:

• [ ] Move KnowledgeBasePage to /features/knowledge-base directory
• [ ] Migrate all knowledge base components to feature-based architecture
• [ ] Implement TanStack Query hooks for knowledge base data fetching
• [ ] Convert to Radix UI primitives where applicable
• [ ] Add proper error boundaries and loading states
• [ ] Colocate tests with components

Architecture Pattern

Following the established patterns from Phase 1 & 2:

• Vertical slice architecture
• TanStack Query for all data fetching
• Smart polling with visibility awareness
• Optimistic updates with rollback
• Radix UI primitives for consistency

Related PRs

• Phase 1: #560 (Projects migration)
• Phase 2: #588 (MCP & Cleanup)

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

[coderabbitai[bot]]

Walkthrough

Consolidates and replaces the legacy Knowledge Base surface: deletes many legacy UI hooks/components/services and the ToastContext, and introduces a feature-scoped Knowledge module (components, hooks, services, types, inspector, progress), backend progress/crawl changes, ETag/API hardening, tests, and integration test config. (50 words)

Changes

Cohort / File(s)	Summary
Build & Test archon-ui-main/package.json, archon-ui-main/vitest.integration.config.ts, archon-ui-main/tests/integration/setup.ts, archon-ui-main/tests/setup.ts, archon-ui-main/vite.config.ts	Add vitest integration config and scripts, test harness/setup, integration config, and Vite proxy/test env adjustments.
App & Toast migration **/src/App.tsx, archon-ui-main/src/contexts/ToastContext.tsx (removed), archon-ui-main/src/features/ui/hooks/useToast.ts, archon-ui-main/src/features/ui/components/ToastProvider.tsx	Remove legacy ToastContext; consolidate toast hook/provider under features/ui and add removeToast; update imports and provider composition.
Removed legacy knowledge UI Deleted: archon-ui-main/src/components/knowledge-base/* (multiple files)	Remove legacy knowledge-base components (AddKnowledgeModal, CrawlingProgressCard, CrawlingTab, DocumentBrowser, EditKnowledgeItemModal, GroupCreationModal, GroupedKnowledgeItemCard, KnowledgeItemCard, KnowledgeItemSkeleton, KnowledgeTable, etc.).
Removed legacy hooks & services Deleted: archon-ui-main/src/hooks/* (useCardTilt, useCrawlQueries, useNeonGlow, useTerminalScroll), archon-ui-main/src/services/api.ts, archon-ui-main/src/services/knowledgeBaseService.ts, archon-ui-main/src/types/* (legacy types)	Remove older UI hooks, legacy service layers and related types.
Knowledge feature — services & types (new) archon-ui-main/src/features/knowledge/services/knowledgeService.ts, .../services/index.ts, .../types/knowledge.ts, .../types/index.ts	Add typed knowledgeService (summaries, detail, crawl, refresh, upload, stop, chunks, code-examples, search) and new knowledge/types exports.
Knowledge feature — hooks (new) archon-ui-main/src/features/knowledge/hooks/useKnowledgeQueries.ts, .../hooks/index.ts, .../hooks/tests/useKnowledgeQueries.test.ts	Add TanStack Query key factory and hooks for summaries/items/chunks/code-examples, mutations (crawl/upload/delete/refresh), progress coordination and polling; include unit tests.
Knowledge feature — components (new) archon-ui-main/src/features/knowledge/components/*, .../components/index.ts	New UI components: AddKnowledgeDialog, DocumentBrowser, KnowledgeCard (+actions/progress), KnowledgeHeader, KnowledgeList, KnowledgeTable, and barrel export.
Inspector (new) archon-ui-main/src/features/knowledge/inspector/components/*, .../inspector/hooks/*, .../inspector/index.ts	Add Knowledge Inspector: ContentViewer, InspectorHeader, InspectorSidebar, KnowledgeInspector, pagination hooks and barrels.
Progress subfeature (new) archon-ui-main/src/features/knowledge/progress/*	Add progress types, progressService, polling/hooks (useOperationProgress, useCrawlProgressPolling, useMultipleOperations), and UI (CrawlingProgress, KnowledgeCardProgress).
Views & page wiring archon-ui-main/src/features/knowledge/views/*, archon-ui-main/src/features/knowledge/index.ts, archon-ui-main/src/pages/KnowledgeBasePage.tsx	Add KnowledgeView / KnowledgeViewWithBoundary, feature index; simplify KnowledgeBasePage to delegate to view.
UI primitives & small UI changes archon-ui-main/src/features/ui/primitives/inspector-dialog.tsx, .../primitives/pill.tsx, .../primitives/button.tsx, various projects/MCP tweaks	Add InspectorDialog primitive, StatPill, Button knowledge variant, accessibility and formatting tweaks.
ETag / API infra hardening archon-ui-main/src/features/projects/shared/apiWithEtag.ts	Robustify ETAG helper: null-safe caches, full-URL cache keys, test-url handling, 10s fetch timeout, improved 304/204/error handling and diagnostics.
Backend API & crawling changes python/src/server/api_routes/knowledge_api.py, python/src/server/api_routes/progress_api.py, python/src/server/main.py, python/src/server/services/crawling/*, python/src/server/services/knowledge/*, python/src/server/services/knowledge/knowledge_summary_service.py, progress types & tracker changes	Add /knowledge-items/summary endpoint, paginate/enrich chunks & code-examples, replace orchestration/background manager patterns with CrawlingService/KnowledgeSummaryService, adopt mapped progress stages and tracker listing/cleanup, update health/startup and progress APIs.
Storage / extraction / progress internals python/src/server/services/storage/*, python/src/server/services/crawling/*, python/src/server/utils/progress/progress_tracker.py, python/src/server/services/crawling/progress_mapper.py	Multiple internal changes: progress mapping overhaul, cancellation handling, document/code extraction progress refinements, deterministic embedding mapping, and progress-tracker delayed cleanup/listing.
Tests & integration/manual scripts archon-ui-main/src/features/testing/test-utils.tsx, archon-ui-main/src/features/knowledge/hooks/tests/*, archon-ui-main/tests/integration/knowledge/*, archon-ui-main/tests/manual/*	Add/adjust unit tests, integration/manual tests for knowledge/progress APIs, and integration test setup/config.
Small UI and tests formatting edits various files (MCP, projects, tests)	Formatting, minor accessibility improvements, re-exports, and test cleanup across multiple modules.

Sequence Diagram(s)

%%{init: {"themeVariables": {"primaryColor":"#7c3aed33","secondaryColor":"#60a5fa22"}} }%%
sequenceDiagram
  autonumber
  actor User
  participant View as KnowledgeView
  participant Dialog as AddKnowledgeDialog
  participant KS as knowledgeService
  participant ProgSvc as progressService
  participant Poll as useCrawlProgressPolling
  participant UI as CrawlingProgress / KnowledgeCardProgress

  User->>View: Click "Add Knowledge"
  View->>Dialog: open()
  alt Crawl website
    Dialog->>KS: crawlUrl(request)
    KS-->>Dialog: { success, progressId }
  else Upload document
    Dialog->>KS: uploadDocument(file, metadata)
    KS-->>Dialog: { success, progressId }
  end
  Dialog-->>View: onSuccess(progressId)
  View->>Poll: register progressId
  loop Polling
    Poll->>ProgSvc: listActiveOperations()/getProgress(progressId)
    ProgSvc-->>Poll: ActiveOperation / ProgressResponse
    Poll-->>UI: update activeOperations / inline progress
  end
  opt Terminal state reached
    UI->>View: onComplete/onError
    View->>KS: invalidate/refetch summaries
  end

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

• coleam00/Archon#567 — TanStack Query / polling refactor; overlaps with new knowledge hooks, query-key patterns, and polling logic.
• coleam00/Archon#564 — DocumentBrowser and chunks/code-examples API changes; overlaps frontend DocumentBrowser and backend chunks pagination.
• coleam00/Archon#514 — Migration from streaming/websockets to HTTP polling; overlaps progress endpoints, polling logic, and progress mapper changes.

Suggested labels

enhancement

Suggested reviewers

• leex279
• coleam00
• TimothiousAI

Pre-merge checks (3 passed)

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title accurately and concisely summarizes the primary change: a Phase 3 TanStack Query migration focused on the Knowledge Base feature, and it directly reflects the large, knowledge-centric changes in the PR. It is specific and readable for teammates scanning history.
Description Check	✅ Passed	The PR description is comprehensive and documents the migration goals, architecture changes, test infrastructure, integration/backend updates, breaking changes, and verification steps, matching the raw summary and objectives. However it does not follow the repository's required template exactly: the explicit "Changes Made" section, the "Type of Change" and "Affected Services" checkbox blocks, the Testing checklist with checkbox marks, and the Test Evidence code block are not filled in using the template headings and checkbox format. Please add those template-specific sections and explicitly mark the checklist items so automated reviewers and maintainers can quickly verify test status and affected services.
Docstring Coverage	✅ Passed	Docstring coverage is 84.06% which is sufficient. The required threshold is 80.00%.

Poem

🐰 I hopped through branches, moved the old to new,
Cards now hum with progress, toasts call out the view.
Inspectors peek at chunks, crawls pulse in line,
Tests and APIs awake — the knowledge gardens shine.
Thump-thump — merge me, please, and let the data dine.

[!TIP]

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

✨ Finishing touches

• [ ] 📝 Generate Docstrings

🧪 Generate unit tests

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch feat/tanstack-migration-phase-3-knowledge-base

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}