Controllable Protein Design by Prefix-Tuning Protein Language Models

Designing novel proteins tailored for specific purposes (e.g. drug discovery, vaccine design) presents a promising approach to address various biomedical challenges. Due to the similarity between protein sequences and natural languages, motivated by the remarkable success in NLP tasks that pre-trained language models have enabled text generation with human-like capabilities, protein language models (ProtLMs) are constructed to generate protein sequences with a predictable function across large protein families. The text generation can be controllable by constructing prefix-phase as control tags to prompt NLP language models. However, the vocabulary of protein sequences only contains 20 amino acid residues, which is not like natural language vocabulary to make up flexible control tags. In this study, we propose a controllable protein design method, named PrefixProt, which utilizes prefix tuning to learn virtual tokens as control tags, enabling to efficiently prompt the pre-trained ProtLM for protein generation tailored for specific purposes. The virtual tokens can be learned on any protein properties by data-driven and are flexible to be combined for fine-grained control. To demonstrate the effectiveness of PrefixProt, we train three virtual tokens on alpha-helix structure dataset, antimicrobial peptide (AMP) dataset and anticancer peptide (ACP) dataset, respectively. Our results show that prefix virtual tokens are efficient to prompt the pre-trained ProtLM by optimizing fewer trainable parameters compared with fine-tuning, especially under low-data settings. When combining the virtual tokens, the proportion of generated proteins with multiple properties are significantly improved. Therefore, PrefixProt offers a flexible and controllable protein design solution. We anticipate that PrefixProt will contribute to drug discovery and biomedical advancement. ### Competing Interest Statement The authors have declared no competing interest.