PodGPT framework. Credit: npj Biomedical Innovations (2025). DOI: 10.1038/s44385-025-00022-0
The rise of generative artificial intelligence (AI), particularly large language models (LLMs), has marked a transformative shift in data analysis, interpretation and content generation. These models, trained on extensive textual datasets, have demonstrated the ability to generate contextually accurate and linguistically rich outputs, with profound implications for fields such as science and medicine, where models like OpenAI's GPT-4 have shown remarkable aptitude.
However, the full potential of LLMs in science, technology, engineering, mathematics and medicine (STEMM) remains underexplored, particularly in integrating non-traditional data modalities such as audio content.
In a new study, researchers from Boston University introduce a newly created computer program called PodGPT that learns from science and medicine podcasts to become smarter at understanding and answering scientific questions. The work is published in the journal npj Biomedical Innovations.
"By integrating spoken content, we aim to enhance our model's understanding of conversational language and extend its application to more specialized contexts within STEMM disciplines," explains corresponding author Vijaya B. Kolachalama, Ph.D., FAHA, associate professor of medicine and computer science at Boston University Chobanian & Avedisian School of Medicine.
"This is special because it uses real conversations, like expert interviews and talks, instead of just written material, helping it better understand how people actually talk about science in real life."
Kolachalama and his colleagues collected more than 3,700 hours of publicly available science and medicine podcasts and turned the speech into text using advanced software. They then trained a computer model to learn from this information.
Following this, they tested the model on a variety of quizzes in subjects like biology, math, and medicine, including questions in different languages, to see how well it performed. The results demonstrated that incorporating STEMM audio podcast data enhanced their model's ability to understand and generate precise and comprehensive information.
According to the researchers, this study shows that voice-based content like podcasts can be used to train AI tools. Kolachalama is also a Founding Member of Faculty of Computing & Data Sciences at Boston University, and an affiliate of Hariri Institute of Computing at Boston University.
"This opens the door to using all kinds of audio, like lectures or interviews, to build smarter and more human-like technology. It also shows promise in making science more accessible in many languages, helping people across the world learn and stay informed," said Kolachalama.
Not only do the researchers believe that this technology will help make scientific and medical knowledge easier to access, but that listening to the conversations of experts in their field will assist people in making more informed decisions about their health and education.
"This could help improve understanding and diagnosis in many health conditions such as Alzheimer's disease, cardiovascular disease, infectious diseases, cancer and mental health. It may also support learning in areas like public health and planetary health," said Kolachalama.
More information: Shuyue Jia et al, PodGPT: an audio-augmented large language model for research and education, npj Biomedical Innovations (2025). DOI: 10.1038/s44385-025-00022-0
Provided by Boston University School of Medicine