Notion

[pipedream-component-development]

App

Notion

Summary:

Can't deploy

Details:

Hi Support,

I’m running into another issue with the Notion “new or updated page” trigger — I can’t deploy the trigger when using a specific database. I suspect it’s related to this warning shown in the component: "Source not saving? Your database might be too large. If deployment takes longer than one minute, an error will occur."

There must be a way around this, as I need to trigger from this particular database, but can't even get the trigger to deploy. Can you please assist?

Here’s the source I’m using — the same one I worked with Andrew on before: https://github.com/PipedreamHQ/pipedream/blob/master/components/notion/sources/updated-page/updated-page.mjs

Thank you, Georges

Screenshots:

No screenshots included

[vunguyenhung]

Hi @andrewjschuang @GTFalcao, I think we really need to revise the Notion - New or Updated Page

[GTFalcao]

@vunguyenhung can we get in touch with the user to see what error they received? The warning mentioned is an alert prop, so it always show up; it'd be helpful to know what error occurs when they try to deploy the trigger.

[michelle0927]

@vunguyenhung I gather that the user hasn't responded. Should this be closed/blocked?

[freivokh]

Hello, how can I help in resolving this? as I still haven't been contacted from support regarding this request and it's still a persistent issue for me. thank you

[vunguyenhung]

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check the test report below for more information https://vunguyenhung.notion.site/Notion-239bf548bb5e8128ab5afaae5c0ffc08

[michelle0927]

@vunguyenhung Getting new events by page property was a user request, so I'd rather not remove that functionality.

I updated the name of the existing source to "New or Updated Page in Database (By Property)", and I created a new source named "New or Updated Page in Database (By Timestamp)". The new source uses the last_edited_time to detect updates, similar to other Notion sources.

[vunguyenhung]

Hi everyone, all test cases are passed! Ready for release!

Test report https://vunguyenhung.notion.site/Notion-239bf548bb5e8128ab5afaae5c0ffc08

[freivokh]

@michelle0927 @vunguyenhung Sorry for my delayed response, and thank you for addressing the fixes. However, I'm still experiencing the same issues. The most valuable part of this Notion trigger is that it activates when a property changes. I'm not sure why you split it into two triggers—by timestamp and by property. I still can't deploy the trigger based on property changes; it now loads indefinitely and won't deploy. I suspect this is related to the size of the Notion database table, but that's my only lead. Thank you again.

[michelle0927]

@michelle0927 @vunguyenhung Sorry for my delayed response, and thank you for addressing the fixes. However, I'm still experiencing the same issues. The most valuable part of this Notion trigger is that it activates when a property changes. I'm not sure why you split it into two triggers—by timestamp and by property. I still can't deploy the trigger based on property changes; it now loads indefinitely and won't deploy. I suspect this is related to the size of the Notion database table, but that's my only lead. Thank you again.

Hi @freivokh. To check for specific property changes, the component needs to retrieve and store all existing values in the database to compare against. I suspect you're correct that having a large database size is causing it to timeout during deploy. The new source that simply evaluates the last_edited_time should execute much quicker, thereby avoiding any timeouts.

I'm not sure if this helps, but Notion does allow creating your own webhooks through their UI: https://developers.notion.com/reference/webhooks#step-1-creating-a-webhook-subscription The payloads for the database.content_updated and page.content_updated event types include an array of updated_blocks.

[freivokh]

Okay thank you very much, I appreciate the information and the additional link. I will try that

[freivokh]

@michelle0927 I looked into the notion webhooks url you shared: It would be incredible if we could set up our custom Notion integrations' webhook subscriptions to work with and integrate with a new Pipedream trigger (with the same filter by property options, etc) specifically made to integrate with these notion webhooks!

Currently, we have so many of these traditional Notion Pipedream triggers polling Notion every 20-30 seconds for changes, that we are constantly running into rate limiting issues. Reported by notion themselves, we are sending about 700k+ requests a week alone from these triggers polling for changes.

If we could transition our Pipedream triggers from the old polling way to a new way which integrates with the new Notion webhooks and listen for changes that way, that would be amazing!

Please let me know your thoughts on this 🙏 A huge amount of our internal company operations relies on Pipedream and these Notion triggers, so this would be huge.

[michelle0927]

Hi @freivokh, I've created and prioritized an issue for developing a webhook-based trigger for retrieving Notion updates. You can track it here: https://github.com/PipedreamHQ/pipedream/issues/18182

[freivokh]

@michelle0927 thank you v much!