Decoding Promoter Activity from DNA Sequence using Pre-trained Language Models

Promoter architecture plays a central role in transcriptional regulation, yet predicting promoter activity directly from DNA sequence remains challenging. Here, we assess whether transformer-based DNA language models can learn and interpret regulatory logic encoded in Drosophila core promoters. We fine-tuned the 117-million-parameter DNABERT-2 model on [~]700 synthetic promoters assayed by [~]2,600 dual-luciferase measurements in Drosophila S2 cells. A sequence-only model achieved high predictive accuracy (R{superscript 2} {approx} 0.91), demonstrating that core promoter sequence alone strongly constrains transcriptional output. Model interpretability using SHapley Additive exPlanations (SHAP) revealed biologically meaningful sequence features corresponding to canonical core promoter elements. Extending the model to incorporate biological context, including Ecdysone (Ecd) hormonal activation and flanking -1/+1 nucleosomal sequences, preserved strong performance while capturing more complex promoter behavior. Gene-wise cross-validation showed robust generalization for most promoters, and application to independent in vivo embryo data demonstrated that the model still generalizes reasonably well, even in complex biological contexts.