Baseline Implementation (VariPred) Model

[merdivane]

No description provided.

[ofivite]

VariPred is one specific solution of fine-tuning, which for a given protein sequence:

1. takes embedding vectors from pretrained PLMs (e.g. ESM) for wildtype and mutated positions in the sequence
2. concatenates them
3. trains simple feedforward network to predict the pathogenicity given the concat vector

I would say, a simpler baseline would be to not train it but rather use some distance metric between wildtype and mutation embedding vectors to see how it correlates with the target (pathogenicity). Similarly as studied in Nucleotide Transformer paper (Fig. 4).

The best performing out of those distance metrics would be our own baseline and the starting point of setting up the pipeline. Then, we can try fine-tuning as VariPred or possibly other strategies to improve upon it.

[AllenChienXXX]

VariPred is one specific solution of fine-tuning, which for a given protein sequence:

1. takes embedding vectors from pretrained PLMs (e.g. ESM) for wildtype and mutated positions in the sequence
2. concatenates them
3. trains simple feedforward network to predict the pathogenicity given the concat vector

I would say, a simpler baseline would be to not train it but rather use some distance metric between wildtype and mutation embedding vectors to see how it correlates with the target (pathogenicity). Similarly as studied in Nucleotide Transformer paper (Fig. 4).

The best performing out of those distance metrics would be our own baseline and the starting point of setting up the pipeline. Then, we can try fine-tuning as VariPred or possibly other strategies to improve upon it.

Do you know the source of this model?

[merdivane]

Nucleotide Transformer
Model weights available here: https://huggingface.co/InstaDeepAI
This model is now available to use with the transformers library! To use, please install from main, i.e. pip install --upgrade git+https://github.com/huggingface/transformers.git
Check out their paper for inspiration: https://www.biorxiv.org/content/10.1101/2023.01.11.523679v1

[ofivite]

I am actually not sure, would their model be useful for us? Because it's for DNA sequences but we do proteins, right?