feat: rewrite docs generation

[lukekarrys]

TODO:

• [ ] Test snapshots for all generated docs content
• [ ] Gitignore only bundled deps

[npm-cli-bot]

no statistically significant performance changes detected

timing results

app-large	clean	lock-only	cache-only	cache-only peer-deps	modules-only	no-lock	no-cache	no-modules	no-clean	no-clean audit
npm@8	46.294 ±1.82	25.841 ±0.58	23.590 ±0.24	26.804 ±1.07	4.051 ±0.10	4.087 ±0.22	3.219 ±0.10	17.082 ±0.52	3.211 ±0.03	4.326 ±0.07
#5309	49.043 ±3.53	27.003 ±0.20	30.165 ±10.80	28.351 ±3.28	3.981 ±0.00	4.034 ±0.10	3.158 ±0.06	16.734 ±0.04	3.162 ±0.05	4.702 ±0.13

app-medium	clean	lock-only	cache-only	cache-only peer-deps	modules-only	no-lock	no-cache	no-modules	no-clean	no-clean audit
npm@8	34.021 ±1.93	18.785 ±0.49	16.566 ±0.13	17.672 ±0.15	3.675 ±0.13	3.566 ±0.14	3.316 ±0.04	12.178 ±0.21	3.078 ±0.01	4.184 ±0.12
#5309	36.846 ±3.08	19.406 ±0.65	17.083 ±0.35	18.100 ±0.35	3.473 ±0.04	3.567 ±0.07	3.173 ±0.06	11.591 ±0.02	3.002 ±0.01	4.194 ±0.10

[lukekarrys]

Since the man and html docs were generated, and we are no longer checking in the generated markdown docs, I did a full diff between the docs in 8.17.0 and in this branch BEFORE I removed all the extra placeholders. The full diff is below and it shows that we are now correctly replacing the versions, adding a simple synopsis for the npm command, and adding two copied man files.

Diff from 8.17.0.tgz

diff --git a/package/man/.DS_Store b/man/.DS_Store
index 63b47e9ed..73cb601a0 100644
Binary files a/package/man/.DS_Store and b/man/.DS_Store differ
diff --git a/package/man/man1/npm.1 b/man/man1/npm.1
index 04133f580..5f2b4a706 100644
--- a/package/man/man1/npm.1
+++ b/man/man1/npm.1
@@ -2,6 +2,12 @@
 .SH "NAME"
 \fBnpm\fR \- javascript package manager
 .SS Synopsis
+.P
+.RS 2
+.nf
+npm
+.fi
+.RE
 .SS Version
 .P
 8\.17\.0

diff --git a/man/man5/npm-global.5 b/man/man5/npm-global.5
new file mode 100644
index 000000000..b6efd3c88
--- /dev/null
+++ b/man/man5/npm-global.5
@@ -0,0 +1,226 @@

diff --git a/man/man5/npm-json.5 b/man/man5/npm-json.5
new file mode 100644
index 000000000..d4f972311
--- /dev/null
+++ b/man/man5/npm-json.5
@@ -0,0 +1,1296 @@

diff --git a/docs/content/.DS_Store b/docs/content/.DS_Store
new file mode 100644
index 000000000..22e638cd1
Binary files /dev/null and b/docs/content/.DS_Store differ

diff --git a/package/docs/content/commands/npm-ls.md b/docs/content/commands/npm-ls.md
index a7936fafc..64719f22c 100644
--- a/package/docs/content/commands/npm-ls.md
+++ b/docs/content/commands/npm-ls.md
@@ -36,7 +36,7 @@ packages will *also* show the paths to the specified packages.  For
 example, running \`npm ls promzard\` in npm's source tree will show:
 
 \`\`\`bash
-npm@@VERSION@ /path/to/npm
[email protected] /path/to/npm
 └─┬ [email protected]
   └── [email protected]
 \`\`\`

diff --git a/package/docs/content/commands/npm.md b/docs/content/commands/npm.md
index 18a68d03c..013459dce 100644
--- a/package/docs/content/commands/npm.md
+++ b/docs/content/commands/npm.md
@@ -7,11 +7,21 @@ description: javascript package manager
 ### Synopsis
 
 <!-- AUTOGENERATED USAGE DESCRIPTIONS START -->
+<!-- automatically generated, do not edit manually -->
+<!-- see lib/commands/npm.js -->
+
+\`\`\`bash
+npm
+\`\`\`
+
+<!-- automatically generated, do not edit manually -->
+<!-- see lib/commands/npm.js -->
+
 <!-- AUTOGENERATED USAGE DESCRIPTIONS END -->
 
 ### Version
 
-@VERSION@
+8.17.0
 
 ### Description
 
diff --git a/package/docs/content/using-npm/package-spec.md b/docs/content/using-npm/package-spec.md
index 0d3741018..fd64131fb 100644
--- a/package/docs/content/using-npm/package-spec.md
+++ b/docs/content/using-npm/package-spec.md
@@ -4,7 +4,6 @@ section: 7
 description: Package name specifier
 ---
 
-
 ### Description
 
 Commands like \`npm install\` and the dependency sections in the
diff --git a/docs/output/.DS_Store b/docs/output/.DS_Store
new file mode 100644
index 000000000..f3ab3515d
Binary files /dev/null and b/docs/output/.DS_Store differ
diff --git a/package/docs/output/commands/npm.html b/docs/output/commands/npm.html
index 523629237..d0bbf8682 100644
--- a/package/docs/output/commands/npm.html
+++ b/docs/output/commands/npm.html
@@ -148,6 +148,12 @@ npm command-line interface
 <div id="_content"><h3 id="synopsis">Synopsis</h3>
 <!-- raw HTML omitted -->
 <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<pre lang="bash"><code>npm
+</code></pre>
+<!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
 <h3 id="version">Version</h3>
 <p>8.17.0</p>
 <h3 id="description">Description</h3>

[wraithgar]

How are we to review changes in usage output now? It was very handy having them in source control to see for PRs. I suppose they still exist in the test snapshots for help/man? Do the full man pages exist in snapshots?

[lukekarrys]

How are we to review changes in usage output now? It was very handy having them in source control to see for PRs. I suppose they still exist in the test snapshots for help/man? Do the full man pages exist in snapshots?

That's a good point. I just checked and we have snapshots for config's describeAll() and shorthands. I'm going to add snapshots that run for all commands to output their usage and params.

Also docs generation will fail if anything couldn't be replaced, so if something isn't being generated properly we will learn about it during CI (or if you run npm run docs locally).

[lukekarrys]

~~There are a lot of commits here with bad messages, but this PR should be squashed.~~ So this is ready for review 😎

Squashed this to add some deps commits that we can update to in the docs workspace, so this should now be rebased.

[lukekarrys]

High level overview of the changes here:

• The source for the docs content has moved from docs/content/ to docs/lib/content/. The generated markdown is still written to docs/content/ but that directory is now ignored from git.
• All generated content sections of the docs have been removed and replaced with single placeholder html comments such as <!-- AUTOGENERATED CONFIG DESCRIPTIONS -->
• Placeholders are replaced with generated content only as part of the prepack step, so generated markdown is no longer checked in to source and all docs related make commands have been removed
• All docs (and docs related) snapshots have been moved to a single test file that outputs command usage and formats it with functions imported from docs/lib/index.js. So tests will fail if docs content changes until npm run snap is run.