PolyFuzz icon indicating copy to clipboard operation
PolyFuzz copied to clipboard

Issue Facing While Fitting The Model With Huge Data

Open ganesh-morsu opened this issue 10 months ago • 5 comments

I have data contains around 166793 Records, I want to fit this records for TF-IDF Model

from polyfuzz.models import TFIDF
from polyfuzz import PolyFuzz

data=[] # Data contains total **166793 Records**
tfidf = TFIDF(n_gram_range=(1, 1), model_id="TF-IDF")
model = PolyFuzz(tfidf)

model.fit(data)

Here i am facing the issue while fitting the model ,The server getting killed (I have tried with configuration of 20 gb ram). Is there any solution?

ganesh-morsu avatar Aug 29 '23 09:08 ganesh-morsu