Can't achive the same score in paper

[kirikokonoka]

I'm also trying to reproduce this paper. The same scores can't be achieved with the same hyperparameters. For example, I can only get 0.81 acc instead of 0.85 in the paper in SearchSnippets dataset. I think it may be the problem of data enhancement. Could you tell me whether the bertbase and roberta in the contextual argument are the original model or not, or what should I do to improve scores?

[huiyegit]

@kirikokonoka I am also interested in this work, but I can not find the dataset to run the source code. How did you get the SearchSnippets dataset?

[urospet]

@huiyegit https://github.com/rashadulrakib/short-text-clustering-enhancement/tree/master/data take a look here for the dataset

[1085737319]

I'm also trying to reproduce this paper. The same scores can't be achieved with the same hyperparameters. For example, I can only get 0.81 acc instead of 0.85 in the paper in SearchSnippets dataset. I think it may be the problem of data enhancement. Could you tell me whether the bertbase and roberta in the contextual argument are the original model or not, or what should I do to improve scores?

I just changed the batch_size to 250, the result was only about 0.2.I don't know what is wrong.

[Dejiao2018]

Can you double check whether there is a bug in your code? Searchsnippets only has 8 classes, so accuracy of 0.2 basically indicates your model only does random cluster assignment.