frame level contextual features

[AswiniKumar1995]

Dear sir,

Can I get all the token-level output contextual features from the model?

[GokulNC]

Hi @AswiniKumar1995 , apologies for late reply.

Yes, you can get token-level output from BERT, but you will have to make changes in the source-code.
This is not available by default in this library, because we currently support only ISLR, which means it aggregates all the frame-level features and tries to classify it to a single sign.

This is the place where the pooling happens:

OpenHands/openhands/models/decoder/bert_hf.py

Lines 97 to 104 in 8824cd0

	if self.pooling_type == "cls":
	x = x[:, 0]
	elif self.pooling_type == "max":
	x = torch.max(x, dim=1).values
	elif self.pooling_type == "avg":
	x = torch.mean(x, dim=1)
	elif self.pooling_type == "att":
	x = self.attn_block(x)