Knowledge MCP Server

A Model Context Protocol (MCP) server for managing and querying knowledge bases through vector databases, built on top of the dynamic-mcp-server (DMS) framework.

Overview

Knowledge MCP Server leverages the dynamic-mcp-server (DMS) as its core, providing a robust, extensible platform for secure, user-aware, and dynamic AI tool servers. All user, tool, and database management is handled by DMS, with extension points for project-specific logic (e.g., knowledge sources, Microsoft auth).

Architecture

• Core Integration: Uses DMS for all authentication, user management, tool registration, and MongoDB/Mongoose connection management.
• Centralized DB Connection: All models and repositories use the shared MongoDB connection provided by DMS.
• User & Tool Management: User and tool models are aligned with DMS, with project-specific extensions via subclassing (see AppUserRepository).
• Custom HTTP Routes: All custom routes (OAuth, health, etc.) are registered on the DMS-exported Express server using DMS APIs (e.g., addAuthHttpRoute).
• Extensibility: Extend user models, repositories, and add new tools or routes by subclassing or using DMS extension points.

Features

• Vector database integration for storing and querying embeddings
• Document processing pipeline for content ingestion
• Text chunking and embedding generation
• Dynamic tool registration and management (via DMS APIs)
• Secure access control and sharing for knowledge sources
• Support for website crawling and content extraction
• MongoDB integration for metadata and vector storage

Prerequisites

• Node.js 18 or later
• MongoDB Atlas M10 or higher instance (for vector search capabilities)
• OpenAI API key

Installation

1. Clone the repository:

   git clone https://github.com/your-org/knowledge-mcp-server.git
   cd knowledge-mcp-server
   

2. Install dependencies:

   npm install
   

Configuration

Create a .env file in the project root with the following variables:

# Server Configuration
PORT=4001
HOST=localhost

# MongoDB Configuration
MONGODB_URI=your_mongodb_connection_string

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key

# Admin Bootstrapping (required by DMS)
MCP_ADMIN_EMAIL=your_admin_email@example.com

Running the Server

Start the server in development mode:

npm run dev

Or in production mode:

npm start

Note: The server is started via DMS, and all custom routes are registered on the DMS-exported Express server. There is no local Express server startup logic.

Tool Registration & Sharing

• All tool registration and sharing is managed via DMS APIs (addTool, sharing endpoints, etc.).
• Legacy tool registration logic has been removed.
• Tools are registered per user/session as needed, supporting dynamic extensibility.

User & Access Model

• User access to knowledge sources is managed via applicationAuthorization.knowledge.owned and applicationAuthorization.knowledge.shared fields on the user record.
• Sharing is handled via the DMS sharing model and APIs, supporting "read" and "write" access levels.
• The user repository is extended as AppUserRepository for project-specific logic (e.g., custom sharing/access methods).

Knowledge Source Management

The server currently supports the following knowledge source type:

1. Microsoft OneDrive
   • Ingests files and folders from OneDrive
   • Supports various document formats (PDF, DOCX, etc.)
   • Requires Microsoft Graph API integration

Note: Website knowledge source support (web crawling, HTML extraction, etc.) is not yet implemented, but is planned for a future release.

Extending the Server

• User Model Extension: Subclass AppUserRepository to add project-specific user logic or fields.
• Custom HTTP Routes: Register new routes using DMS APIs (e.g., addAuthHttpRoute).
• Tool Registration: Add new tools using DMS's addTool and related APIs.
• Model/Repository Extension: Prefer subclassing or wrapping DMS repositories for custom logic.

Available Tools

The server provides the following tools:

1. Add Knowledge

• Tool Name: add-knowledge
• Purpose: Ingests and processes new documents into a knowledge source
• Functionality: Accepts document sources (URLs, text, or file uploads), scrapes/processes content, chunks and embeds into a vector database, associates with a knowledge source

2. Search

• Tool Name: search
• Purpose: Queries a specific knowledge source for relevant information
• Functionality: Semantic search against the vector database, returns relevant document fragments with metadata, supports filtering/ranking

3. Use Knowledge Source

• Tool Name: use-knowledge-source
• Purpose: Creates a new tool for interacting with a specific knowledge source
• Functionality: Generates a new tool definition, configures with parameters/constraints, enables dynamic tool registration

4. Refresh Knowledge Source

• Tool Name: refresh-knowledge-source
• Purpose: Refreshes a knowledge source by re-ingesting its content
• Functionality: Updates status, clears vector store, re-processes source, updates status

Access Control & Sharing

• Ownership: Users own knowledge sources they create and have full access.
• Sharing: Knowledge sources can be shared with other users ("read" or "write" access), tracked in applicationAuthorization.knowledge.shared.

Dependencies

The project uses the following key dependencies:

{
  "@llm-tools/embedjs": "latest",
  "@llm-tools/embedjs-mongodb": "latest",
  "@llm-tools/embedjs-openai": "latest",
  "@llm-tools/embedjs-loader-web": "latest",
  "mongodb": "latest",
  "dynamic-mcp-server": "github:scitara-cto/dynamic-mcp-server"
}

Testing

• Run the test suite:

  npm test
  

• Run tests in watch mode:

  npm run test:watch
  

• Type checking:

  npm run typecheck
  

• Mocking: Use the mocking strategies and test patterns recommended in the DMS documentation. Tests should use the extended repositories (e.g., AppUserRepository).

Migration Status

• All migration and refactor steps are complete except for documentation.
• All tests and build pass.
• The codebase is fully aligned with the new DMS architecture and repository model.

License

[Your License Here]