Peptone AI team presents our Sentence Transformers research during NeurIPS 2023 Conference

Petpone's Director of Applied Machine Learning and AI, Dr. Istvan Redl, discussed with Oliver Dutton, the Head of Numerical Modelling, at Peptone headquarters in London, UK.

Protein language models (pLMs) have been widely used in in-silico protein engineering tasks and have shown remarkable results.

However, their application methods are mostly standardized. In this regard, our team introduced a set of fine-tuning techniques based on Sentence Transformers (STs) that were integrated with a novel data augmentation procedure. Our team demonstrated how this can lead to new state-of-the-art performance.

 

 Schematic overview of finetuning pLMs using Sentence Transformers.

Although STs were initially developed in classic NLP space, they hold natural appeal in pLM-related applications, mainly due to their use of sequence pairs and triplets in the process.

Example of how a finetuned Sentence Transformer alters the embeddings.

Our team showcased this conceptual approach in two different settings that frequently occur in this domain: a residue and sequence-level prediction task.

Apart from demonstrating how these tools can extract more and higher quality information from pLMs, our team will discuss the main differences between their applications in NLP and protein spaces.

Peptone's researchers are investigating novel methods of structural disorder modelling at our UK headquarters in London, UK.

More about this research.

Download the NeurIPS research paper.

Read more about the Biology Track at NeurIPS 2023.

About Peptone

Peptone is a translational biophysics company focused on the discovery of novel therapeutics against diseases driven by intrinsically disordered proteins. IDPs are proteins without a fixed structure that play a significant role in health and disease.