henhaobio
Sign in

Journal of genetics and genomics/Journal of Genetics and GenomicsPaleontologyPeer-reviewed journal article

Rapid and robust sex determination from ancient enamel proteomes using protSexInferer

Fan BaiiD, Zhongyou WuiD, Xia SongiD, Qiaomei FuiD

PDFDOIPreprintCodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Accurate biological sex determination of ancient remains is critical for archaeological, anthropological, and forensic studies, but remains challenging for morphologically ambiguous and highly degraded endogenous DNA samples. Paleo-proteomics sex identification approaches, targeting sexually dimorphic amelogenin isoforms (AMELX and AMELY), present a promising solution. However, current workflows rely on manual verification of a few specific peptide markers, a process that lacks standardization and is susceptible to false-positive AMELY signals. To overcome these limitations, we developed protSexInferer, a lightweight, open-source bioinformatic pipeline for automated sex estimation from paleo-proteomic data. Our method uses the ratio of AMELY-specific peptides to all detected AMELY- and AMELX-specific peptides (i.e., the RAMELY value) rather than the mere presence or absence of AMELY signals for sex classification. We demonstrated that the RAMELY value clearly distinguishes male and female individuals in both reference and independent validation datasets, enabling reliable sex assignments even in cases where conventional intensity-based comparisons (e.g., AMELY-59M vs. AMELX-60) are ambiguous. This ratio-based approach effectively mitigates the impact of false-positive AMELY signals, therefore eliminating the need for time-consuming manual verification, and remains reliable even for samples with low peptide yields. Equipped with pre-constructed protein reference databases, protSexInferer provides a robust, standardized, and end-to-end solution for paleo-proteomic sex determination.

In-depth summary

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 1 (25 Mar 2026) · CC BY 4.0

Using the share of AMELY-specific among all sex-specific amelogenin peptides (RAMELY), enamel proteomes are sexed consistently across four search engines, tolerating false-positive AMELY hits without manual checks.

Amelogenin-based paleo-proteomic sexing works on fragmented, subadult or DNA-free teeth, but existing workflows hinge on one or a few hand-picked dimorphic peptides (such as AMELY 59M) and expert manual checks to rule out spurious AMELY matches. protSexInferer replaces presence/absence calling with a peptide-count ratio over all AMELX/AMELY variant sites, bundled with prebuilt reference databases and parsers for PEAKS, pFind, MaxQuant and DIA-NN.

Key results

  1. ShownAcross 76 known-sex reference samples, RAMELY values split into two sex-specific clusters for each of the four supported search engines, although absolute ranges differed by engine; PEAKS gave the widest male-female gap and was made the default.

    Per-engine thresholds are set from the 95% CI limits of the female maximum and the male minimum (Table S3). MaxQuant gave the lowest thresholds and DIA-NN (DDA mode) the highest. Default PEAKS thresholds: male RAMELY > 0.088, female RAMELY < 0.055. · The effectiveness of protSexInferer’s sex…; Step 3: Sex determination and report generation

  2. ShownOn 69 published individuals previously sexed by proteomics, spanning adult and non-adult (including prenatal) individuals, deciduous and permanent teeth, and specimens from present-day to about 2 Ma, RAMELY calls matched the prior assignments in almost all cases with every search engine.

    Agreement is shown per engine in Figure 3; the text gives no count of discordant samples. Individual JY63, morphologically assessed as male, was called female, matching the earlier proteomic call. · The effectiveness of protSexInferer’s sex…; Sample collection

  3. ShownSpurious AMELY-specific peptides appear in some females even after the reference database was adjusted. An intensity rule comparing AMELY-59M with AMELX-60 (Madupe et al.) turns these into false male calls, which the ratio-based thresholds avoid.

    The Madupe et al. method misclassified one female as male with MaxQuant and gave numerous false male calls with PEAKS and DIA-NN (DDA mode); RAMELY reassigned these individuals as female. pFind lacks intensity output and was not compared. The authors expect AMELY false positives modestly above the nominal PSM FDR of 0.010. · The robustness of protSexInferer to the false…

  4. SuggestedRAMELY changes little with how many amelogenin peptides a sample yields. Using the full-length AMELY sequence lets AMELY-specific peptides other than 59M rescue calls when 59M peptides are faint or missing.

    R^2 = 0.053 (p = 0.004) in males and R^2 = 0.110 (p = 0.007) in females with RAMELY > 0 (Figure S1). Samples VG3_2 and VG19_2, whose 59M-related peptide intensities were below 10% of typical levels, were assigned correctly via additional AMELY-specific peptides. · The robustness of protSexInferer to the false…

  5. SuggestedThe choice of reference database matters. Two smaller databases gave results like the default and fit its thresholds, whereas a larger mammalian database produced more false positives and did not fit the default thresholds.

    Tested with PEAKS on 10 randomly selected individuals: the Human enamel proteins and Hominidae AMEL databases matched the default Hominidae enamel protein database; the Mammalian enamel proteins database showed a notably higher false-positive rate, so the authors recommend custom thresholds for any database larger than the default. · Sex determination with other databases; Customization

Methods, data and reproducibility

protSexInferer (Nextflow) parses PEAKS, MaxQuant, pFind or DIA-NN (DDA) output, keeps amelogenin peptides unique to AMELX or AMELY after cross-checking the reference database, computes RAMELY = nAMELY/(nAMELY + nAMELX) with a normal-approximation 95% CI, and calls sex against per-engine thresholds from known-sex samples. Calibrated on 76 published samples, validated on 69 and applied to 8 new teeth.

  • Known-sex reference enamel samples from previously published studies (Table S1), used to set RAMELY thresholds · n = 76 samples
  • Published validation enamel samples with prior proteomic sex calls (Table S2): adult and non-adult, deciduous and permanent, archaeological (up to ~2 Ma) and present-day · n = 69 individuals
  • Newly analyzed scattered teeth from the Xiawanggang site, Henan, China (ca. 5,000 BP); acid-etched enamel, LC-MS/MS on Orbitrap Exploris 480 · human · n = 8 teeth · PXD072753 (ProteomeXchange)
  • Random subset of individuals used to compare three alternative reference databases with PEAKS · n = 10 individuals
  • Default Hominidae enamel protein reference database (enamel proteins including AMELX, AMELY, AMBN, ENAM, MMP20) · Pongo, Gorilla, Pan, Homo, Gigantopithecus, Paranthropus
Methods & tools
protSexInferer, Nextflow, Conda, PEAKS, MaxQuant, pFind, DIA-NN, Orbitrap Exploris 480, Easy n-LC 1200, C18 ZipTips, ProteomeXchange / PRIDE
Software & environment
Nextflow pipeline; dependencies resolved through Nextflow's Conda integration (no admin rights needed). Supported inputs: PEAKS v.11+, MaxQuant v.2.6.0.0+, pFind v.3.2.1+, DIA-NN v.2.3.0+ in DDA mode (Homo samples only). Pipeline and Nextflow versions not stated.
Reproducibility gaps
The user must run the database search first. Search parameters, database contents, per-engine thresholds (Table S3) and sample metadata (Tables S1, S2) are in supplementary files not included here. PXD072753 is listed with reviewer login details.
Limitations (6)
  • Males lacking the AMELY gene or its expression would be missed, which the authors note is very rare. · stated by the authors
  • AMELY-specific peptides can be absent by chance, particularly in highly mineralized samples or with insufficient sampling. · stated by the authors
  • Only applies to mammals carrying both AMELX and AMELY isoforms (including primates), not to species such as mice that have a single AMEL(X) isoform. · stated by the authors
  • Default thresholds may not hold for custom or larger databases or changed search parameters; the Mammalian enamel proteins database raised false positives, so thresholds should be re-estimated from known-sex samples. · stated by the authors
  • Ground truth for the 69 validation individuals is earlier proteomic sex calls rather than DNA or skeletal sex, and agreement is described only as almost all cases, with no stated count of discordant samples. · noted in this summary
  • Thresholds rest on the single most extreme female and male values among 76 reference samples, so one outlier can shift them; the 8 Xiawanggang male calls have no independent sex check. · noted in this summary

Read this if you sex archaeological, fossil or forensic teeth from enamel LC-MS/MS data and want an automated, threshold-based alternative to manually checking AMELY peptide hits.

Citations
1
FWCI
4.05
Field percentile
top 9%

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISMorvan '26Bray '26Koenig '24Kotli '26Fabrice '20Ahmad '21
84% semantic matchEvolutionary Biology
Marine Morvan·bioRxiv·Jun 9, 2026· 1 citations