Load markdown from external sources

[samholmes]

I plan on using docsify for repo docs. One of the requirements I have is to have one repo link out to each repo's docs index file (README.md). This main repo will be the org-wide repo with a table of contents of all other documentation as well as organization/high-level documentation. The goal is to publish this org-wide docs repo and have it be the primary knowledge source for the organization which would include each repo's documentation on that website.

In order to do this, it would make sense if I can link to other external README.md files by URL. Right now docsify reads the directory to create table of contents and URL paths. Could it also grep (using ripgrep for fast file search) for all markdown links and source paths/pages from external? One could configure a baseUrl map so it only looks for links including a baseUrl in the map. The baseUrl table could allow for developers to map baseUrls to some overrides for local development and testing with the locally cloned repos.

[samholmes]

Below is an example of a theoritical readme tool which has a design which meets my stated requirements. This can be used as an example of how a tool can achieve the results of what I need, but it is in no way a proposal for a redesign for docsify.

Instead, this is can be used to understand the goal better and perhaps include features in docsify to accomplish the same goal.

ReadMe CLI Tool Guide

Overview

The ReadMe CLI tool is designed to automatically generate and preview documentation from Markdown files with minimal setup. Its primary focus is to help developers quickly convert their Markdown documentation into navigable, local HTML previews for easy review and sharing.

It also offers advanced multi-repository support—enabling seamless integration and previewing of documentation that references resources across multiple repositories.

Key Features

• Automatic Documentation Generation: Instantly converts Markdown files into structured, browsable HTML documentation.
• Minimal Configuration: No config files required; simply use CLI flags to get started.
• Multi-Repo Support: Scans local directories for git repositories, matches them to GitHub links in your Markdown files, and can substitute remote links with local paths for cohesive previews.
• Ambiguity Handling: Chooses the best local match for each link, with logging and optional user prompts.
• HTML Generation: Creates a single HTML page with a sidebar index for easy navigation.

How It Works

1. Process Markdown

The tool parses specified Markdown files and prepares them for preview.

2. Grepping for Links

The ReadMe CLI scans each Markdown file line-by-line using a regular expression (/\[([^\]]+)]\(([^)]+)\)/g) to find every Markdown link or image reference. For each link, it captures:

• The link text (e.g., Install guide)
• The raw URL or relative path (e.g., https://github.com/acme/installer/blob/main/README.md)
• The byte offset in the file for efficient rewriting

This streaming approach ensures fast processing, even for large documentation files.

3. Building a Dynamic Table of Contents

Every heading (#, ##, ###, etc.) is registered in an in-memory outline during scanning. After link substitution, this outline is converted into an HTML table of contents and injected into the sidebar of your preview. Each entry points to an auto-generated anchor, keeping navigation in sync with your content.

4. Generate HTML

Processed Markdown is converted into a single HTML file with sidebar navigation.

5. Invented URL Path Structure & Local Serving

The CLI mounts a static server (default http://localhost:4000) and exposes every processed Markdown file under a predictable URL scheme:

/preview/<repo-name>/<branch-or-tag>/<relative-path>.html

For example:

• Main project README: http://localhost:4000/preview/my-project/main/README.html
• Referenced file from a sibling repo: http://localhost:4000/preview/utils-lib/develop/docs/configuration.html

Links in the HTML are rewritten to these paths, so clicking between pages keeps you on the local server.

6. Browse

Open the root URL (/preview/<repo>/<branch>/README.html) to explore your docs with live internal navigation, local links, and an always-accurate sidebar.

Usage Examples

1. Basic Usage

Generate an HTML preview from a Markdown file:

readme-cli --file ./README.md --output ./preview.html

2. Multi-Repo Substitution

Process a README and substitute GitHub links with local paths using clones found in ~/dev:

readme-cli --local ~/dev --file ./README.md --output ./preview.html

3. Prompt for Ambiguous Matches

Prompt the user to select which local repo to use if multiple clones match a remote:

readme-cli --local ~/dev --file ./README.md --prompt

4. Verbose Logging

Print detailed logs about repo detection, substitutions, and ambiguities:

readme-cli --local ~/dev --file ./README.md --verbose

5. Process Multiple Markdown Files

Process and preview multiple Markdown files:

readme-cli --local ~/dev --files ./README.md ./CONTRIBUTING.md --output ./preview.html

6. Help/Usage

Display usage information and available flags:

readme-cli --help

Flags/Options Reference

• --local <dir>: Directory to scan for local git repos and override external links.
• --file <file> / --files <files>: Markdown file(s) to process.
• --output <file>: Output HTML file.
• --prompt: Prompt user if multiple local clones match.
• --verbose: Enable detailed logging.
• --help: Show help.

Substituting External Links with the --local Feature

The --local flag is a powerful feature that enables developers to override external GitHub resource links with local clones for development and preview purposes. When you specify a directory with --local, the tool scans for local git repositories and builds a mapping between GitHub URLs and your local filesystem paths.

Whenever a Markdown link points to a GitHub repository that matches a local clone, the tool substitutes the remote link with a local path. This allows you to preview your documentation as if all referenced resources were available locally, making it easy to review and test documentation changes across multiple projects before pushing updates to GitHub.

The substitution respects ambiguities by preferring directories that match the project name or are less nested, and can prompt you to choose the correct repository if multiple matches exist.

This feature is especially useful for:

• Local Development: Previewing how your documentation will look and function with local versions of referenced resources.
• Multi-Repo Projects: Ensuring internal links between documentation in different repositories work seamlessly during development.
• Testing: Validating documentation changes across multiple projects in a unified, local environment.

Example: Programmatic API (Node.js)

If you want to use the tool programmatically:

Typical Workflow

1. Clone all related repositories into a single directory (e.g., ~/dev).
2. Run the CLI tool with the --local flag pointing to that directory.
3. The tool automatically finds local clones, matches them to GitHub links, and substitutes those links in your documentation.
4. Optionally, generate an HTML preview for easy browsing.
5. If there are ambiguities, use the --prompt flag to choose the correct repo.

Why Use This Tool?

• Instantly preview Markdown documentation as structured HTML.
• Test documentation changes across multiple projects before pushing to GitHub.
• Override external links for local development and review.
• Keep your workflow lean—no config files, minimal flags, and smart automation.

---

Ready to streamline your documentation workflow? Try the ReadMe CLI tool and make multi-repo docs easy to manage and preview!