[Bug] X static feed in auto grabbed mode fetches duplicate posts due to commented code

[kirikak2]

Bug Description

When using X (Twitter) static feed in auto grabbed mode, the same posts are repeatedly fetched. This creates duplicate posts in the feed.

Root Cause

I identified the issue in the code. The problem appears to be in the static/x.js file where an entire block of code is commented out:

https://github.com/steveseguin/social_stream/blob/main/sources/static/x.js#L354

/* try {
   var msglink = ele.querySelector("a[href] > time").parentNode.href;
   if (autoGrabTweets) {
       if (grabbedTweets.includes(msglink)) {
           return;
       }
   }
   grabbedTweets.push(msglink);
   grabbedTweets = grabbedTweets.slice(-250);
   localStorage.setItem('grabbedTweets', JSON.stringify(grabbedTweets));
} catch (e) {
   if (autoGrabTweets) {
       return;
   }
} */

This commented-out block contains the logic that should prevent duplicate posts. Because this section is commented out, posts that are auto-grabbed are not being recorded, causing the system to fetch the same posts repeatedly.

Reproduction Steps

1. Configure Social Stream to use X static feed
2. Enable auto grabbed mode
3. Observe the feed for several minutes
4. Notice the same posts appearing multiple times

Fix Verification

I tested uncommenting this code block, and this resolved the issue. Posts are now only fetched once as expected.

Additional Note

I've also noticed that X feeds have all posts wrapped in tags in the output. Enabling the "Strip HTML/images from messages and donations" option helps somewhat, but it causes style issues with external links. Is this a known issue or should I create a separate bug report for it?

[steveseguin]

Thank you for identifying this issue and for providing such a nice issue ticket.

I'm not entirely sure why I commented that out, but I suppose I was troubleshooting and forgot to re-enable it. I can't see why re-adding it would cause any issues, so I'll do that. Thank you. I've pushed an update to the beta branch that reverted that.

As per your additional note, I made some changes for you to try -- hopefully I was able to clean up some of the code. You can also try with &striphtml or &striplinks , to see if those handle the links better.

I've not heavily tested it, and I'll need to review it some more in the morning, but hopefully it helps.

Thank you