engine

engine copied to clipboard

Define how to track multi-page documents

32

# Context and Problem Statement Some documents are divided into several sub-documents accross many web pages. For example, the Community Guidelines for [Twitter](https://help.twitter.com/en/rules-and-policies) or [Facebook](https://transparency.fb.com/fr-fr/policies/community-standards/) are divided, whereas those for...

Ndpnt

I think it could be interesting that the linter actually catches careless errors, e.g., forgetting to close a string, or forgetting a coma.  

Amustache

When i run `npm start ` command for services who begin to have a lot of documents (for exemple: 15 for Instagram [here](https://github.com/OpenTermsArchive/france-elections-declarations/blob/main/declarations/Instagram.json), 15 for Twitter [here](https://github.com/OpenTermsArchive/france-elections-declarations/blob/main/declarations/Twitter.json)) it's took a...

clementbiron

- [ ] Check if a remote exist before trying to push with the GitAdapter It will allow to start the engine with GitAdapter config `publish` set to true even...

Ndpnt

Improve documents validation message

7

Declaration validation does not consider an empty `documents` key as an error. ### What happened I tried to validate with `npm run validate Example` this malformed declaration: ```json { "name":...

MattiSG

- [ ] Update package metadata. - [ ] Document semver practice in CONTRIBUTING. - [ ] Add publishing to CD. - [ ] Add automatic validation that the version...

MattiSG

Extract terms through OCR for non-text source documents

6

With the following declaration (the dedicated branch is here https://github.com/OpenTermsArchive/declarations-france/commit/5d1c1c3bb1e43ad68e86b5ed346090d3c29e78f3 ) ``` { "name": "Desigual", "documents": { "Commercial Terms": { "fetch": "https://www.desigual.com/on/demandware.static/-/Library-Sites-DsglSharedLibrary/default/dw98507c8d/docs/legal/Footer_legal_documents/Francia/FRANCIA-Condiciones_Generales_Venta_Vfinal_FR_230321.pdf" }, "Privacy Policy": { "fetch": "https://www.desigual.com/on/demandware.static/-/Library-Sites-DsglSharedLibrary/default/dw77e5bf6a/docs/legal/Footer_legal_documents/Francia/FRANCIA-POLITICA_DE_PRIVACIDAD_Vfinal_FR_230321.pdf" } }...

clementbiron

Bypass bot detectors

10

Hi, Rakuten and Leboncoin have very strong bot detectors, hence preventing from automatically fetching their CGUs (at least on a regular OVH machine). See https://fr.shopping.rakuten.com/newhelp/conditions-generales/ or https://www.leboncoin.fr/dc/cgu. It is possible...

LVerneyPEReN

Use server metadata to determine if document changed

2

To optimize http requests, it would be good to send If-Modified-Since and/or If-None-Match headers and use ETag and Last-Modified.

michielbdejong

Add Ad Publishing Privacy Policy type

9

I can't detemine the type of this document : https://privacy.acuityads.com/technology-privacy-policy.html Should we have to add `Ad Publishing Privacy Policy` to available types ?

clementbiron