sitediff fails with "Not a directory @ apply2files" if crawl only produces one page

[fgerards]

Sitediff fails to compare 2 single-paged URLs/sites: the before/after entries in the snapshots directory are files, not directories containing other entries, so this should also be taken into account

Error occurs on Linux ubuntu laptop and on Macbook Air 2020 M1 with MacOS Ventura 13.1 when installing sitediff via homebrew in latest version

[jgam]

any follow up comment on this?

[kirk-brown-ew]

Which version of Ruby are you using? Can you provide an example of what you're doing? On what line does the error happen?

[jgam]

using ruby 3.1.3

when running sitediff crawl it simply finds only a single path that is '/' and outputs the error above

[jgam]

for in stnce this is the how output looks like

Jimmyui-MacBook-Pro:~ jimmygam$ sitediff init https://mentree.club/
[success] Created /Users/jimmygam/sitediff/sitediff.yaml
Jimmyui-MacBook-Pro:~ jimmygam$ sitediff crawl
Reading config file: /Users/jimmygam/sitediff/sitediff.yaml
Visited https://mentree.club/, cached.
[error] Unknown parsing error for https://mentree.club/: Not a directory @ apply2files - sitediff/snapshot/before/timestamp  From page: {:referrer=>"/"}

1 page(s) found.
[done] Created /Users/jimmygam/sitediff/paths.txt.

[kirk-brown-ew]

We've been able to reproduce this issue by:

1. Creating a static web page with no links.
2. Running sitediff crawl.
3. Adding a link to the page with either a reference to within the page or creating another page at the same level.
4. Running sitediff crawl.

The first crawl creates the before site as a file. The second crawl wants to create before as a directory.

The solution that we see is to remove the before directory and re-running the crawl.

[tjhaygood]

Adding on to @kirk-brown-ew's comment, I've been able to resolve this issue entirely by running the following before running the crawl command:

mkdir -p sitediff/snapshot/before ; mkdir -p sitediff/snapshot/after

Your path before the /snapshot directory may differ. I'm running this via the Docker image.