Bug in few-shot sampling process

[prohor33]

Hi!
It seems like function

Few-NERD/util/fewshotsampler.py

Line 62 in 3935544

def __valid_sample__(self, sample, set_class, target_classes):

has a bug, which affects sampling process.

You can see "for loop" and 4 "if" inside it. If flag isvalid is set to True or False in some iterations, it can be overwritten in next iterations of the loop. So, for example, first class breaks the rule shots_count <= 2*K, but second class is new and flag isvalid will be returned as True from __valid_sample__, but sample breaks validation rules.

Thus, with this bug, we can sample samples, which are breaking sampling rules, e.g. with shots_count > 2*K.

What do you think? Does this bug affects results somehow? Maybe in the good way even.

[mtt1998]

Hi, I just find the released episodes for train/dev/test contain episodes that have support count > 2*K shot.

[yulinchen99]

Thanks for reminding! We have noticed the bug here and it does lead to some ">2*K shot" issues. We will fix the bug soon. The influence should be small, however, since it is not a frequent case.

[prohor33]

@cyl628 Hi! Why you closed the bug? Did you fix the bug?

[iofu728]

Hi folks,
@cyl628 Thanks for your work, but I find the Few-NERD web page also uses the wrong link in https://github.com/ningding97/fewnerd/blob/d173373b8a28f170912ff5681798b312fe75ec85/_layouts/home.html#L113.
It's misleading. Could you fix it?

Thanks!