Multimodal Machine Learning Applications

Towards Video-LLM Driven Workflow for Behavioral Segmentation and Scoring in Mice Performing a Skilled Water Reaching Task: An Evaluation of Recent LLM Models

The paper evaluates video LLMs for automated behavioral segmentation and scoring in mice performing a water-reaching task, finding that Gemini 2.5 Pro achieves high accuracy with specific resolution and visibility requirements.

Modality-Agnostic Decoding of Vision and Language from fMRI

This paper introduces modality-agnostic decoders for fMRI data to predict stimuli regardless of input modality, showing they perform as well as or better than modality-specific decoders.

CALM-VLM: CALIBRATION AND SELECTIVE PREDICTION IN VISION-LANGUAGE MODELS FOR RELIABLE BRAIN MRI CLASSIFICATION

CALM-VLM improves confidence and reliability in brain MRI classification by calibrating vision-language models and enabling selective prediction.

Does Human-Like Contextual Object Recognition Emerge from Language Supervision and Language-Guided Inference?

This study investigates how scene context influences object recognition in humans and models, showing that vision-language models with language supervision better replicate human performance.

Descriptron: Testing Artificial Intelligence for Automating Taxonomic Species Descriptions with a User-friendly Software Package

Descriptron uses a Vision Language Model to automate taxonomic species descriptions with error correction, but requires taxonomist supervision to prevent hallucinations.

Leveraging a Vision-Language Model with Natural Text Supervision for MRI Retrieval, Captioning, Classification, and Visual Question Answering

This paper introduces a framework using a vision-language model with natural text supervision to perform MRI retrieval, captioning, classification, and visual question answering in Alzheimer's disease research.

Egocentric perception of walking environments using an interactive vision-language system

This paper develops a multimodal vision-language system for egocentric perception of walking environments, generating detailed image captions with user personalization.

Assessing large multimodal models for one-shot learning and interpretability in biomedical image classification

This study evaluates large multimodal models for one-shot learning and interpretability in biomedical image classification, showing their superiority over traditional methods.

SciMind: A Multimodal Mixture-of-Experts Model for Advancing Pharmaceutical Sciences

SciMind is a multimodal model designed to handle pharmaceutical sciences data, showing good performance in text comprehension and molecular tasks.

CELL-E 2: Translating Proteins to Pictures and Back with a Bidirectional Text-to-Image Transformer

CELL-E 2 translates protein sequences to images of subcellular localization and vice versa, enabling de novo protein design and generating novel nuclear localization signals.