TokenScope

What is TokenScope?

TokenScope is a Model Context Protocol (MCP) server for token-aware directory exploration and analysis. It helps Large Language Models (LLMs) understand codebases by scanning directory structures with token-efficient summaries, extracting file contents with token awareness, finding important files, and generating comprehensive reports.

How to use TokenScope?

Install via uv pip install tokenscope (Python 3.10+ required). Run with the mandatory --base-path argument to restrict all file operations to a specific directory. For Claude Desktop, add the server to mcpServers in the configuration file (e.g., ~/.config/claude/config.json), providing the uv run command with required arguments. A development installation option from GitHub is also available.

Key features of TokenScope

• Token-aware directory scanning with configurable depth and .gitignore support
• Smart file content extraction respecting token limits
• Token usage statistics broken down by file extension
• Markdown report generation with structure and token statistics
• File selection prioritization based on importance
• Search files by pattern within directory structure

Use cases of TokenScope

• Scanning a project directory to quickly understand its structure and key files
• Estimating token consumption for processing a codebase with an LLM
• Generating a comprehensive directory report including token statistics and file samples

FAQ from TokenScope

What is the mandatory argument for running TokenScope?

The --base-path argument is mandatory. It restricts all file operations to the specified directory for security; attempts to access files outside this path are rejected.

How do I configure TokenScope in Claude Desktop?

Add the server to the mcpServers section of Claude Desktop's configuration file. Example: "TokenScope": { "command": "uv", "args": ["run", "--with", "tokenscope", "tokenscope", "--base-path", "/your/secure/base/path"] }.

What are the dependencies and runtime requirements?

Python 3.10 or higher is required. The recommended package manager is uv. For development, fastmcp and tiktoken are additional dependencies.

Does TokenScope store any data or communicate externally?

The README does not mention any data storage or external communication. All operations are file-system based within the allowed base path.

What MCP tools does TokenScope provide?

It provides six tools: scan_directory_structure, extract_file_content, search_files_by_pattern, analyze_token_usage, copy_file_to_destination, and generate_directory_report.