AI for IACUC: Accurate Initial Assessment of Institutional Animal Care and Use Committee Protocols

Abstract

Every animal protocol requires careful review to ensure humane principles, laws, and policies are followed before any experiments may be performed. Developing protocols and receiving approval can be a lengthy process requiring multiple rounds of feedback and refinement. Artificial Intelligence (AI) has the potential to improve the quality and speed of IACUC protocol reviews by detecting submission errors. We describe an approach to achieve this and show that our approach can be replicated, while addressing issues of ethics and bias, robustness, and trustworthiness1. Replication is addressed by developing a template and demonstrating that it can be reused. Ethics and bias issues are addressed by early, limited use of a large language Model (LLM) to identify errors and rapidly provide useful feedback to the protocol developer. In our approach, subsequent steps still require a review by the IACUC board, which retains ethical responsibility for approval. Robustness and trustworthiness were evaluated by using the template that is replicated for eleven distinct checks and performed on 50 new protocols using two different LLMs. We show that every actual problem in those submissions (within the test parameters) is found by our AI review (i.e., recall=100%), giving IACUC boards confidence in the utility of our system. In addition, precision for our checks is between 80%-100%, with most being 100%, thereby making the system a practical compliment to administrative protocol review for those submitting. Overall, we report F1 scores of between 89-100%.