goclone
goclone copied to clipboard
Can we control the cloning depth? (e.g., clone only the homepage)
Question:
Hi! I’m exploring GoClone for cloning specific pages (e.g., cloning just the homepage of https://apple.com without recursively downloading the entire site). Does GoClone support limiting the crawling depth?
For example:
- A
--depthflag to restrict recursion (e.g.,--depth=1for only the homepage). - Excluding internal links beyond the root path.
If this isn’t possible today, could you suggest a workaround or confirm if it’s on the roadmap?
Use Case:
I need to clone a website for testing/styling without pulling gigabytes of unrelated content.
Thanks for building this tool!
Do you have an example of logs where the recursion can be seen?
I tested and dont see recursive
goclone https://www.apple.com/
Extracting --> https://www.apple.com/
Css found --> /api-www/global-elements/global-header/v1/assets/globalheader.css
Extracting --> https://www.apple.com/api-www/global-elements/global-header/v1/assets/globalheader.css
Css found --> /ac/globalfooter/8/en_US/styles/ac-globalfooter.built.css
Extracting --> https://www.apple.com/ac/globalfooter/8/en_US/styles/ac-globalfooter.built.css
Css found --> /ac/localnav/9/styles/ac-localnav.built.css
Extracting --> https://www.apple.com/ac/localnav/9/styles/ac-localnav.built.css
Css found --> /wss/fonts?families=SF+Pro,v3|SF+Pro+Icons,v3
Extracting --> https://www.apple.com/wss/fonts?families=SF+Pro,v3|SF+Pro+Icons,v3
Css found --> /v/home/cd/built/styles/main.built.css
Extracting --> https://www.apple.com/v/home/cd/built/styles/main.built.css
Css found --> /autopush/us/fam-gallery/fam-gallery.built.css
Extracting --> https://www.apple.com/autopush/us/fam-gallery/fam-gallery.built.css
Css found --> /ac/ac-films/7.0.0/styles/modal.css
Extracting --> https://www.apple.com/ac/ac-films/7.0.0/styles/modal.css
Js found --> /metrics/ac-target/latest/ac-target.js
Extracting --> https://www.apple.com/metrics/ac-target/latest/ac-target.js
Js found --> /v/home/cd/built/scripts/head.built.js
Extracting --> https://www.apple.com/v/home/cd/built/scripts/head.built.js
Js found --> /api-www/global-elements/global-header/v1/assets/globalheader.umd.js
Extracting --> https://www.apple.com/api-www/global-elements/global-header/v1/assets/globalheader.umd.js
Js found --> /metrics/ac-analytics/2.24.0/scripts/ac-analytics.js
Extracting --> https://www.apple.com/metrics/ac-analytics/2.24.0/scripts/ac-analytics.js
Js found --> /autopush/us/fam-gallery/fam-gallery.built.js
Extracting --> https://www.apple.com/autopush/us/fam-gallery/fam-gallery.built.js
Js found --> /ac/globalfooter/8/en_US/scripts/ac-globalfooter.built.js
Extracting --> https://www.apple.com/ac/globalfooter/8/en_US/scripts/ac-globalfooter.built.js
Js found --> /ac/localeswitcher/4/en_US/scripts/localeswitcher.built.js
Extracting --> https://www.apple.com/ac/localeswitcher/4/en_US/scripts/localeswitcher.built.js
Js found --> /v/home/cd/built/scripts/main.built.js
Extracting --> https://www.apple.com/v/home/cd/built/scripts/main.built.js
Js found --> /ac/ac-films/7.0.0/scripts/autofilms.built.js
Extracting --> https://www.apple.com/ac/ac-films/7.0.0/scripts/autofilms.built.js
Js found --> /metrics/data-relay/1.1.4/scripts/data-relay.js
Extracting --> https://www.apple.com/metrics/data-relay/1.1.4/scripts/data-relay.js
Js found --> /metrics/data-relay/1.1.4/scripts/auto-relay.js
Extracting --> https://www.apple.com/metrics/data-relay/1.1.4/scripts/auto-relay.js