awesome-infograhics icon indicating copy to clipboard operation
awesome-infograhics copied to clipboard

Published 20 hours ago • verified publisher icon sara-02

Find a way to detect duplicate entires.

Open sara-02 opened this issue 7 years ago • 1 comments

  • One Easy way to start will be check for duplicate URLs
  • Check for similarity among title

sara-02 avatar Feb 08 '18 15:02 sara-02

load everything into pandas and remove duplicates. Very Fast ! (Specially for small data)

antiDigest avatar Feb 10 '18 04:02 antiDigest