classifai icon indicating copy to clipboard operation
classifai copied to clipboard

Added "Recommended Content" Block

Open iamdharmesh opened this issue 3 years ago • 9 comments
trafficstars

Description of the Change

This PR Adds the Initial "Recommended Content" Block backed by the Azure Personalizer. Major changes of PR are listed below.

  • Added @wordpress/scripts
  • Format js and fixed reported eslint errors.
  • Integration with Azure Personalizer API.
  • Added "Recommended Content" Block.

Note: This is yet initial implementation based on personalizer documentation and we may still have to check on the below stuff to close #69

  • Include User location data in contextFeatures for better personalization.
  • Currently, block only suggest 1 recommended content as personalizer only return 1 rewarded action. We should check on the possibility/workaround to display multiple contents. (We are showing multiple items now, as decided in internal slack discussion)
  • Define suggested configuration to make the plugin work more effectively (config like model update frequency, reward wait time, etc...).
  • Verify content personalization with huge content and different use cases

Some Useful links for Azure Personalizer:

Screenshots

Personalizer Settings:

image

Recommended Content Block:

image

Closes #69

Alternate Designs

We may choose any other recommended content service, other than Azure Personalizer.

Possible Drawbacks

Not identified any.

Verification Process

  1. Save Azure Personalizer settings.
  2. Verify it gets authenticated and saved. (wrong key we should show error on save)
  3. Add "Recommended Content" Block
  4. Verify Filters and various setting options work properly.
  5. Verify Block content displays properly in front-end and send the reward to azure on click-on content.

Checklist:

  • [x] I have read the CONTRIBUTING document.
  • [x] My code follows the code style of this project.
  • [ ] My change requires a change to the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] I have added tests to cover my change.
  • [ ] All new and existing tests passed.

Changelog Entry

Added: Initial "Recommended Content" Block

Credits

Props @iamdharmesh @jeffpaul @Sidsector9

iamdharmesh avatar May 11 '22 13:05 iamdharmesh

Currently, block only suggest 1 recommended content as personalizer only return 1 rewarded action. We should check on the possibility/workaround to display multiple contents.

If we're unable to find a way to get multiple posts recommended back from Azure Personalizer, then we should consider a slight renaming from ~Recommended Content (and related copy) to something more akin to ~Read This Next (since it would only be 1 post recommended).

jeffpaul avatar Jun 03 '22 02:06 jeffpaul

@dkotter pinging you here as reminder on code review / feedback on this PR / new functionality.

jeffpaul avatar Jun 30 '22 16:06 jeffpaul

I've left a few comments on the code but haven't actually tested this out yet.

But prior to that, I do think this warrants a conversation on what we actually want to achieve here. As noted, we can only return a single piece of content, so not sure a full-fledged Recommended Content block makes sense here. Jeff mentioned this could be used as a Up Next sort of feature, and that probably works better. The issue I'm seeing though is this feature is meant to be used on a small content base (maximum of 50 items). We are putting that limit in place at the code-level but I wonder how that impacts results on sites that have hundreds or thousands of posts.

An intriguing idea that was mentioned as a use-case in the documentation is determine what product to suggest to shoppers. I could see us building an integration with WooCommerce for a feature like that, which may be more useful than a Read Next section. Another use-case mentioned is figure out the optimal position for an advertisement which also could have wide usage.

So I don't want to de-rail things here and this may be useful enough to release and try out, but I wonder if there are better options for this Personalizer service, realizing we can only show a single item and we can only have a limited set of content (max 50). Also Has a minimum of ~1k/day content-related events for Personalizer to be effective, so will need to mention this will only work properly on sites that get decent traffic.

dkotter avatar Jul 01 '22 16:07 dkotter

Following up on my last comment, we discussed this more internally and while I think there might be better use cases for Personalizer (especially due to some of the limitations) I do think it's worth getting this particular feature released and see if we can gather some real-world feedback.

A couple additional thoughts/questions:

  1. We'll want to update the README.md file with information about this new feature (and probably the caveats). We'll want to call out this as being a new feature and I think we'll want steps on how to get this setup, since you need to create a Personalizer service instead of a Computer Vision service.
  2. Do we want to rename this feature or are we fine with Recommended Content? I think we'll definitely want to update the block description to make it clear that it only renders a single item
  3. Should we output some sort of heading within the block markup? Right now we just render the one content item based on the settings chosen. Wondering if we want some sort of Recommended heading to make a clear distinction between this and the regular content. Or do we just want to rely on users adding that themselves?
  4. I'm curious how this works on sites that get little to no traffic. For instance on my local, how does it decide which piece of content to show? Is it just random out of the 50 items we send? Seems I get the same post each time (even after clearing localstorage). And I notice in the admin preview, anytime I change a setting it re-renders the block and seems to show me a new post each time. I could see this confusing people, not sure if we need some sort of way to make it clear that the content shown in the admin is just an example and may not be the same as the item that gets shown on the front-end?

dkotter avatar Jul 06 '22 17:07 dkotter

Sorry, one other question. When someone clicks on a recommended content item, we send that information to Personalizer. Is that the only time we send information to them? I guess just curious if there's additional information/tracking we should be sending to them so we get better results.

dkotter avatar Jul 06 '22 17:07 dkotter

@dkotter Thanks for a detailed review here (Sorry for some formatting issues, not sure why those issues are not detected by phpcs). I have updated the codebase as per the review feedback.

  1. We'll want to update the README.md file with information about this new feature (and probably the caveats). We'll want to call out this as being a new feature and I think we'll want steps on how to get this setup, since you need to create a Personalizer service instead of a Computer Vision service.

Sure, @jeffpaul can you please help to azure account details. So I can write down the steps for creating an account and grabbing the keys from there. we may also have to provide some information to users on the suggested configuration to make the plugin work more effectively as I mentioned in PR description.

Do we want to rename this feature or are we fine with Recommended Content? I think we'll definitely want to update the block description to make it clear that it only renders a single item

Yes, agree. We should update the name to something like Read This Next as suggested by @jeffpaul, any suggestion from your side @dkotter ?

  1. Should we output some sort of heading within the block markup? Right now we just render the one content item based on the settings chosen. Wondering if we want some sort of Recommended heading to make a clear distinction between this and the regular content. Or do we just want to rely on users adding that themselves?

We can add a heading as a part of the block, but I followed what core blocks do here, like Latest posts, Query Loop etc... it only render a list of posts without heading. Also, keeping the heading separate from the block and allow users to add heading will be better as per me, because it will give them more control on modifying the style, size, etc... of the heading.

  1. I'm curious how this works on sites that get little to no traffic. For instance on my local, how does it decide which piece of content to show? Is it just random out of the 50 items we send? Seems I get the same post each time (even after clearing localstorage). And I notice in the admin preview, anytime I change a setting it re-renders the block and seems to show me a new post each time. I could see this confusing people, not sure if we need some sort of way to make it clear that the content shown in the admin is just an example and may not be the same as the item that gets shown on the front-end?

I am not sure on behind scene process of how the personalizer decided the rank of the article and didn't find any docs on that as well. maybe we can add notice at some place in block?

When someone clicks on a recommended content item, we send that information to Personalizer. Is that the only time we send information to them? I guess just curious if there's additional information/tracking we should be sending to them so we get better results.

Personalizer only has reward API to send data about user interaction (which also only accept 1 parameter). So, apart from reward API we don't have any other API to send user data. API Doc: https://westus2.dev.cognitive.microsoft.com/docs/services/personalizer-api/operations/Rank

This demo https://personalizercontentdemo.azurewebsites.net/ has a similar use case (shows a personalized article) with some information on how it works, API request, and response information.

Please let me know if you need any further information on this.

Thanks

iamdharmesh avatar Jul 07 '22 12:07 iamdharmesh

This is looking good to me, thanks for all the work here @iamdharmesh! Looks like some merge conflicts that need resolved.

Also, I think still needing updates to our readme around this new feature and how to get it set up. I think you may be waiting on something from @jeffpaul, at least according to this comment.

dkotter avatar Jul 22 '22 19:07 dkotter

Thanks for the review @dkotter, I am made some minor improvements and resolved conflicts.

Also, I think still needing updates to our readme around this new feature and how to get it set up. I think you may be waiting on something from @jeffpaul, at least according to https://github.com/10up/classifai/pull/343#issuecomment-1177521049.

I have updated readme based on docs available online, maybe @jeffpaul can help to verify it once.

Do you think we should release this feature as Alpha/Beta for now by adding notice somewhere? also, We have some further stuff (mentioned in PR description) to work on it once this get merged.

Thanks.

iamdharmesh avatar Jul 29 '22 13:07 iamdharmesh

I think this is good to go as a first version of this.

Do you think we should release this feature as Alpha/Beta for now by adding notice somewhere

We could note this in the README, that this is a feature that we're still gathering feedback on, but not sure if that's critical

dkotter avatar Aug 05 '22 18:08 dkotter

@iamdharmesh I just did a first though thorough pass on copy and such, but would love to test this out a bit more closely. Mind spinning this PR out into a test site for me to see if there are other places to tweak the UX/copy/etc?

jeffpaul avatar Aug 31 '22 02:08 jeffpaul