What I've Been Reading
open source AI devices, Apple's MM1 Models & Digital Twins
Video:
Open Interpreter 01 Light: The open-source foundation for this new era of AI devices By Open Interpreter
What it’s about
This unlocks a massive opportunity for open source devs to build in the application layer as we evolve research for AI agents and task based agents. A true AI assistant or second pair of hands seem fast approaching reality
I’m most excited about to see how firms build applications in enterprise healthcare/lifescience spanning care coordination and clinical R&D.
Interview with Killian Lucas from Alex Volkov:
Reading:
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training By Brandon McKinzie, Zhe Gan et.al (Apple)
What it’s about
Insight into some of the GenAI research going on at Apple:
“In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons. For example, we demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state- of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published multimodal pre-training results. Further, we show that the image encoder together with image resolution and the image token count has substantial impact, while the vision-language connector design is of comparatively negligible importance. By scaling up the presented recipe, we build MM1, a family of multimodal models, including both dense variants up to 30B and mixture-of-experts (MoE) variants up to 64B, that are SOTA in pre-training metrics and achieve competitive performance after supervised fine-tuning on a range of established multimodal benchmarks”
A Systematic Literature Review of Digital Twin Research for Healthcare Systems: Research Trends, Gaps, and Realization Challenges By DOULOTUZZAMAN et. al
What it’s about
The review identifies current research trends, including global interest and interdisciplinary collaborations to address complex healthcare system challenges, but finds existing research predominantly focuses on conceptualization while research on integration, verification, and implementation is nascent.
The authors identify two key research gaps - considering the human-in-the-loop nature of healthcare systems with a focus on provider decision-making, and the need for more implementation research - as well as realization challenges related to improving virtual-to-physical connectivity and addressing data collection, synthesis, and privacy issues.
Generative artificial intelligence empowers digital twins in drug discovery and clinical trials By Steve Cheney
What it’s about
A high-level overview of what a Digital Twin is, some technical architectures folks in the space are using to build applications in this space, and shortcomings of approaches