Submission to Archive.org

[libering]

can i use it to output Archive.org Permalink?

Also would you like to add the support of saving dynamic website?

[oxij]

can i use it to output Archive.org Permalink?

No, this project archives everything to your local machine only.

Also would you like to add the support of saving dynamic website?

The extension saves everything, so it's replay of such websites that is flaky.

It can be done via

hoardy-web serve --expr 'response.body|eb|scrub response &all_refs,+scripts' <inputs>

which will work for very simple websites, but XMLHttpRequest and fetch APIs will still try to fetch from the original URLs, which is why this option is not the default.

Also, I prefer DOM snapshots to re-playing JavaScript anyway, since DOM snapshots can be grepped easily.

[libering]

can i use it to output Archive.org Permalink? No, this project archives everything to your local machine only. i'm prefer to have a online archive for a safety reason

[libering]

You can don't use this function but cannot without it

[oxij]

Well, if you don't want to use the alternatives that implement those functions, you are welcome to add them here.

I'll happily accept code contributions that do exporting to WARC and a hoardy-web sub-command for submitting WARCs to archive.org.

[libering]

Well, if you don't want to use the alternatives that implement those functions, you are welcome to add them here. i don't use the alternatives such as archivebox because i do'nt understand how to set the configuration file

I'll happily accept code contributions that do exporting to WARC and a hoardy-web sub-command for submitting WARCs to archive.org. `async function initiateInternetArchiveUpload(archiveData, filename, metadata) { const backendUploadUrl = 'YOUR_BACKEND_UPLOAD_URL'; // Replace with your backend URL

const payload = { filename: filename, metadata: metadata, archiveData: archiveData, // Ensure this is in a format your backend can handle (e.g., Blob, ArrayBuffer, or a string representation) };

try { const response = await fetch(backendUploadUrl, { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(payload), });

const responseData = await response.json();

if (response.ok) {
  console.log('Upload initiated successfully:', responseData);
  // Update UI to show success
} else {
  console.error('Failed to initiate upload:', response.status, responseData);
  // Update UI to show error
}

} catch (error) { console.error('Error communicating with backend:', error); // Update UI to show error } }

// Example usage within hoardy-web const archiveData = /* Get the archive data from hoardy-web */; const archiveFilename = 'hoardy-web-archive.wrr'; const uploadMetadata = { title: 'Hoardy-Web Archive', description: 'Archive created using Hoardy-Web', // Add other relevant metadata };

// Call the function to initiate the upload // initiateInternetArchiveUpload(archiveData, archiveFilename, uploadMetadata);`

`// WARNING: Directly embedding your secret key in client-side code is a security risk!

async function uploadDirectlyToInternetArchive(fileData, filename, accessKey, secretKey, metadata) { const endpoint = 's3.us.archive.org'; // Replace with the actual endpoint if different const bucketName = 'your-internet-archive-bucket'; // Replace with your bucket name const objectKey = filename; const url = https://${endpoint}/${bucketName}/${objectKey};

// This is a very simplified example and lacks proper signature generation. // You would need to use an S3 signing library here.

const headers = { 'Content-Type': 'application/octet-stream', 'Authorization': AWS ${accessKey}:YOUR_SIGNATURE_HERE, // Replace with actual signature 'x-archive-meta-title': metadata.title || '', 'x-archive-meta-description': metadata.description || '', // Add other metadata headers as needed (refer to IA documentation) };

try { const response = await fetch(url, { method: 'PUT', headers: headers, body: fileData, });

// ... (rest of the error handling as in the previous example)

} catch (error) { console.error('Error during upload:', error); } }

// Example usage (replace with your actual data and keys) const archiveData = /* Your hoardy-web archive data */; const archiveFilename = 'hoardy-web-archive.wrr'; const yourAccessKey = 'YOUR_ACCESS_KEY'; const yourSecretKey = 'YOUR_SECRET_KEY'; const archiveMetadata = { title: 'Hoardy-Web Archive', description: 'Archive created using Hoardy-Web', };

// Call the direct upload function (USE WITH CAUTION) // uploadDirectlyToInternetArchive(archiveData, archiveFilename, yourAccessKey, yourSecretKey, archiveMetadata);`

these are create from AI,it may helpful

[libering]

https://docs.google.com/document/d/1Z28ffBfXCbPWhObscHqDoncGE7JYnTC93cGaeqIYN74/edit?usp=sharing it maybe useful