Defining and Evaluating Cell-Cell Relation Extraction from Biomedical Literature under Realistic Annotation Constraints

Abstract

Extracting cell-cell relations from biomedical literature is essential for understanding intercellular communication in immunity, inflammation, and tissue biology. However, cell-cell relation extraction has not been established as a standalone biomedical relation extraction task, and no benchmark corpus or systematic evaluation framework currently exists. Fully manual corpus construction is costly and difficult to scale, limiting literature-based analyses of cell-cell communication. Here, we define a sentence-level cell-cell relation extraction task and construct complementary manually annotated corpora under realistic annotation constraints. To enable scalable annotation, rule-based literature mining is used solely as an annotation accelerator to identify candidate sentences, while all relation labels are assigned manually. In addition, an independently annotated PubMed corpus without rule-based filtering is constructed to evaluate robustness on natural sentence distributions. Using these resources, we evaluate representative model configurations involving entity indication strategies, classification architectures, and continued pre-training. Our results show that cell-cell relation extraction remains challenging under realistic conditions. Increasing training data size yields consistent performance gains, and specific combinations of entity-aware architectures and continued pre-training provide modest robustness improvements. Nevertheless, performance on unfiltered PubMed sentences remains in the 70% accuracy range, and error analyses indicate that failures cannot be readily explained by simple surface-level factors. Comparisons with general-purpose large language models further suggest that task complexity, rather than model class, is the primary limiting factor. Together, these findings establish a practical foundation for literature-scale cell-cell relation extraction while clarifying its intrinsic limitations.