Fix deserializing bad input third attempt

[ido777]

Background

Fixes #778, surrogates errors

Changes

1. added lib to requirments.txt
2. changed the insertion of the text to the list to be fixed before adding it to the list
3. added test of with problematic test that creates an encoding error but pass the addition to the cache with no error

Documentation

Test Plan

Added dedicated test

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			May 2, 2023 5:38am

[p-i-]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[netlify[bot]]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	ac208f2f51a12c4614303aca6feadad79c068001
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/64b1c1e0ecef44000851d29f

[codecov[bot]]

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.02 :tada:

Comparison is base (376ecf0) 51.00% compared to head (ac208f2) 51.02%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3666      +/-   ##
==========================================
+ Coverage   51.00%   51.02%   +0.02%     
==========================================
  Files         118      118              
  Lines        4884     4886       +2     
  Branches      643      643              
==========================================
+ Hits         2491     2493       +2     
  Misses       2212     2212              
  Partials      181      181              

Impacted Files	Coverage Δ
autogpt/memory/vector/memory_item.py	86.66% <100.00%> (+0.30%)	:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[Pwuts]

Better late than never. :')

Thanks @ido777!