DanR

Revisiting HDBSCAN tuning and topic clustering

20

This is a long post and I apologize going in. I also want to make clear that none of this should be read as a criticism of BERTopic or the...

reduce_topics assigns many documents to -1

18

From what I can see from both experience and in the code `reduce_topics()` reassigns to `-1` frequently. Is this the expected behavior? If I'm understanding the overall picture, topic clusters...

Extract Topic Assignments from Model

2

It seems that if a model is created without specifying `nr_topics` then `BERTtopic.hdbscan_model.labels_` will return the initial assignments. Reading through the code it looks like that when `BERTopic.reduce_topics()` is called...

Too many tokens

3

Hi @jalammar, With my dataset I'm getting > CohereError: too many tokens: total number of tokens (prompt and prediction) cannot exceed 2048 - received 6354. Try using a shorter prompt...