Protein sequence profile prediction using ProtAlbert transformer

Protein profiles have many applications in bioinformatics. To construct the profile from a protein sequence, the sequence is aligned with database. However, sometimes there are no similar sequences with the query. This paper proposes a method based on pre-trained ProtAlbert transformer to predict the profile for a single protein sequence without alignment. The performance of transformers on natural languages is impressive. Protein sequences can be viewed as a language; therefore, we can benefit from using these models. We analyze the attention heads in different layers of ProtAlbert to show that the transformer can capture five essential protein characteristics of the family from a single protein sequence. These assessments are performed on the CASP13 dataset to find representative heads for each of five protein characteristics. Then, these heads are investigated on one thermophilic and two mesophilic proteins as case studies. The results show the significant attention heads for protein family properties extracted from a single protein sequence. This analysis led us to propose an algorithm called PA_SPP for profile prediction using only a single protein sequence as input. In our algorithm, we apply the masked language modeling method of ProtAlbert. The results display high similarity between the predicted profiles and HSSP profiles.