PIAS Lab retweetledi
🚨 New research: Can LLMs be used for 𝐬𝐞𝐪𝐮𝐞𝐧𝐭𝐢𝐚𝐥 𝐝𝐞𝐜𝐢𝐬𝐢𝐨𝐧-𝐦𝐚𝐤𝐢𝐧𝐠 in complex, ambiguous environments?
Our new paper — Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning — shows the answer is yes, and the gains are substantial.
Most AI decision-making research assumes the world is fully observable and unambiguous. Real-world problems — especially in domains such as healthcare — are neither. Our framework fine-tunes a pretrained LLM (Llama-2-7B) on offline, oracle-labeled trajectories so it can tackle MDPs, POMDPs, and Ambiguous POMDPS (APOMDPs): settings with partial observability and model ambiguity.
💡 Key findings:
→ SFT slashes the optimality gap from 43% → 15% vs. random baselines in long-horizon MDPs
→ In partially observed and ambiguous environments, fine-tuned LLMs outperform ICL-only baselines and DPT by up to 15 percentage points
→ On the Darkroom navigation task, our model achieves ~95% of oracle performance
→ Robust to out-of-distribution conditions — strong generalization without retraining
📐 On the theory side, we interpret the fine-tuned attention mechanism as implicitly estimating optimal Q-functions, and derive an end-to-end suboptimality bound that cleanly separates in-context estimation error from training-length bias.
This matters deeply for use in areas such as healthcare, where experimentation is costly or unethical and offline observational data are abundant. Our goal: give clinicians AI systems that reason under genuine uncertainty — not just clean textbook settings.
Huge congratulations to my current and former lab members Minmin Zhang and @aghaei_sina on this work!
📄 Read the paper: arxiv.org/abs/2605.09009
#ArtificialIntelligence #MachineLearning #LLM #ReinforcementLearning #Healthcare #DecisionMaking #Research #Sequential
English
