Endowing Protein Language Models with Structural Knowledge

1 code implementation • 26 Jan 2024 • Dexiong Chen, Philip Hartout, Paolo Pellizzoni, Carlos Oliver, Karsten Borgwardt

Drawing from recent advances in graph transformers, our approach refines the self-attention mechanisms of pretrained language transformers by integrating structural information with structure extractor modules.

Language Modelling Masked Language Modeling +2