Miguel Sozinho Ramalho

refactor code to separate API logic from core.py logic

5

**Description** > TLDR: I (and surely others) cannot use unfurl as a python library but refactoring code.py would fix that. The current [core.py](https://github.com/obsidianforensics/unfurl/blob/main/unfurl/core.py) contains both the logic needed to do...

make browsertrix-crawler runnable in serverless environments

10

Hi all, I've been experimenting with making an AWS lambda function for browsertrix-crawler and I've gone some distance but hit a snag that the maintainers are probably better equipped to...

how configurable is the Automated Profile Creation feature

1

The [automated profile creation](https://github.com/webrecorder/browsertrix-crawler/issues/449) feature is super useful but as I understand [the current implementation](https://github.com/webrecorder/browsertrix-crawler/blob/c3b98e5047ea219336883b0b1969da425fc43456/create-login-profile.js#L224-L236) is somewhat limited in terms of the expected login form names. For instance, it does...

applies to telethon_archiver and tbot_archiver the logic in https://github.com/bellingcat/auto-archiver/blob/601572d76e19d24ce1c3e8881b15d23a4bea987e/src/auto_archiver/archivers/telethon_archiver.py#L102-L105 (for telethon) is not being properly called everytime and so ghost .session files are left behind.

Add a feature which can be enabled/disabled in the screenshot_enricher that allows for a full page PDF capture of the page, in addition to the screenshot itself.

improve config parsing to accommodate type casting and validation rules

1

As it stands the logic to handle the `configs()` method of each Step (like Archivers, Enrichers, ...) does not cast int/float/bool and uses only strings. Likewise it's proven quite useful...

use https://github.com/webrecorder/warcio.js to avoid extracting media for html file

2

Since the wacz_enricher can sometimes have hundreds of media items it does not scale well, one alternative is to dynamically fetch those using JS in the generated hmtl file, this...

feat: add estimation of explosion time, if applicable to video/audio archived

1

There's a cool new tool https://github.com/davidnewschool/sound-delay which should be tested with some examples and if proven reliable can be added as an enhancer. The idea is if a video/audio is...

add docker-compose and document wacz-enricher changes

1

follow up for https://github.com/bellingcat/auto-archiver/pull/93/files

Some results include things like `&` instead of `&` due to, I believe, the scraping mechanism.