notion-py icon indicating copy to clipboard operation
notion-py copied to clipboard

Notion Webclipper functionality

Open 5hay opened this issue 5 years ago • 19 comments

The webclipper is convenient because it also saves the content of the page.

I'd love to be able to programmatically request Notion to parse and save the contents of links into a Notion table.

Background: I want any link I store into my Pocket account to also be available in my Notion list (especially the content).

5hay avatar Apr 04 '19 12:04 5hay

This would be a great candidate for a community contribution, if anyone wants to take a stab! I haven't played with the webclipper functionality much yet myself, or looked at the backend data structures. Happy to answer questions, though, if anyone wants to take it on.

jamalex avatar May 07 '19 23:05 jamalex

I'll try my hand at it later today.

Looks like the endpoints are getWebClipperData, searchWebClipperPages, and addWebClipperURLs

Penagwin avatar Jul 12 '19 16:07 Penagwin

In simplest way looks like requests.post can be used

curl -X POST -H "Content-Type: application/json" --cookie "token_v2="YOURTOKEN" -d '{"type":"block","blockId":"PAGE_WHERE_WEBCLIPPEDPAGE_APPEAR","items":[{"url":"https://gist.github.com/subfuzion/08c5d85437d5d4f00e58","title":"curl examples"}],"from":"curl"}' https://www.notion.so/api/v3/addWebClipperURLs

Will get you response: {"parentBlock":{"record":{"table":"block","type":"page","id":"PAGEID","role":"editor","name":"NAME","iconEmoji":"📲"},"ancestors":[]},"createdBlockIds":["CREATEDBLOCKID"]}

Data pass formatted:

{
    "type": "block",
    "blockId": "BLOCK_WHERE_PAGE_APPEAR",
    "items": [
        {
            "url": "URL",
            "title": "TITLE"
        }
    ],
    "from": "ANYTHING?"
}

Url https://www.notion.so/api/v3/addWebClipperURLs

Whole page structure renders in Notion correctly. I guess backend is responsible to that. Not sure if block id can be nested page

Blucknote avatar Sep 13 '19 14:09 Blucknote

Thank you for providing the raw curl command. Unfortunately it doesn't seem to work, I'm getting the following response:

{ 
   "errorId":"RANDOM ID",
   "name":"ValidationError",
   "message":"Invalid record request"
}

5hay avatar Sep 16 '19 11:09 5hay

in "blockId" I passed top-level page.id gotten from:

page = client.get_block("https://www.notion.so/PAGE")
page.id
'99999999-9999-9999-9999-999999999999'
{
    "type": "block",
    "blockId": "99999999-9999-9999-9999-999999999999",
    "items": [
        {
            "url": "https://www.postgresql.org/docs/current/functions-json.html",
            "title": "result page"
        }
    ],
    "from": "ANYTHING?"
}

https://www.notion.so/result-page-85559943128b4c14803f7d1de15017a3

Blucknote avatar Sep 16 '19 17:09 Blucknote

Oh my bad, somehow I thought the blockId is just the url part (in your example PAGE). Thank you for clarifying!

5hay avatar Sep 17 '19 09:09 5hay

Did anybody make any progress on adding this to the library? Let me know if you have any questions!

jamalex avatar Jan 01 '20 18:01 jamalex

in "blockId" I passed top-level page.id gotten from:

page = client.get_block("https://www.notion.so/PAGE")
page.id
'99999999-9999-9999-9999-999999999999'
{
    "type": "block",
    "blockId": "99999999-9999-9999-9999-999999999999",
    "items": [
        {
            "url": "https://www.postgresql.org/docs/current/functions-json.html",
            "title": "result page"
        }
    ],
    "from": "ANYTHING?"
}

https://www.notion.so/result-page-85559943128b4c14803f7d1de15017a3

hello, I have tried what you say, but still get response like: { "errorId":"RANDOM ID", "name":"ValidationError", "message":"Invalid record request" }

May I ask if your code still work? And can you tell me if https://www.notion.so/PAGE in " page = client.get_block("https://www.notion.so/PAGE") " must be a database page or normal page? I was confused

Thanks!

realCrush avatar Aug 19 '20 06:08 realCrush

Take a look on how I have implemented it myself https://github.com/elblogbruno/NotionAI-MyMind

elblogbruno avatar Aug 19 '20 07:08 elblogbruno

I've been trying to make this curl command work but got responses like :

"name":"ValidationError","message":"Invalid input.
"name":"ValidationError","message":"User cannot edit block."

Turned out

  • "from":"curl" is currently invalid - "from":"chrome" works
  • block id should be formatted with dashes - using pageid copied directly from URL won't work (my bad)

Thanks @elblogbruno for code sample.

amariusz avatar Nov 05 '20 12:11 amariusz

curl -X POST -H "Content-Type: application/json" --cookie "token_v2="YOURTOKEN" -d '{"type":"block","blockId":"PAGE_WHERE_WEBCLIPPEDPAGE_APPEAR","items":[{"url":"https://gist.github.com/subfuzion/08c5d85437d5d4f00e58","title":"curl examples"}],"from":"curl"}' https://www.notion.so/api/v3/addWebClipperURLs

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

TitusRobyK avatar Jun 01 '21 12:06 TitusRobyK

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

Blucknote avatar Jun 01 '21 12:06 Blucknote

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

Hey Thanks btw, But for me Inspect views : background.html is not coming , when i open manage extensions

image

May be they might have done something to hide it ... not sure though

TitusRobyK avatar Jun 01 '21 12:06 TitusRobyK

Hey Thanks btw, But for me Inspect views : background.html is not coming , when i open manage extensions

May be they might have done something to hide it ... not sure though

No problem. I left Chrome a while, found on SO this answer which might help. May be it's under details?

Blucknote avatar Jun 01 '21 12:06 Blucknote

@TitusRobyK can you try this method? chrome-extension://<extension-id>/background.html Found from this article

Blucknote avatar Jun 01 '21 13:06 Blucknote

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

Thank a lot @Blucknote

TitusRobyK avatar Jun 01 '21 13:06 TitusRobyK

chrome-extension:///background.html

Nope this did not work , Even checked the Network call as well , upon adding an article via Notion Web Clipper , I guess its some sort of new implementation i think ( i am not sure though )

image

TitusRobyK avatar Jun 01 '21 13:06 TitusRobyK

Hi, it seems that the Notion web clipper does not use background html, but it uses popup html(and the file name is index.html). So you can check network calls via inspection of popup html.

image

wns3645 avatar Jun 13 '21 12:06 wns3645

Hi, it seems that the Notion web clipper does not use background html, but it uses popup html(and the file name is index.html). So you can check network calls via inspection of popup html.

image

thank you !!!!! Never noticed it

TitusRobyK avatar Jun 13 '21 13:06 TitusRobyK