Size Wu

🔥 We release Harmon: a unified framework for multimodal understanding & generation with a shared visual encoder (vs. decoupled Janus/-Pro). 💥 SOTA on GenEval, MJHQ, WISE 🧠 Strong understanding performance 📄 Paper: huggingface.co/papers/2503.21… 🔗 Code: github.com/wusize/Harmon

✅ Harmon models are fully converted to 🤗 HuggingFace format—easy to load and integrate into your pipelines. 👉huggingface.co/wusize/Harmon-…

🔥 We release Harmon: a unified framework for multimodal understanding & generation with a shared visual encoder (vs. decoupled Janus/-Pro). 💥 SOTA on GenEval, MJHQ, WISE 🧠 Strong understanding performance 📄 Paper: huggingface.co/papers/2503.21… 🔗 Code: github.com/wusize/Harmon

🖼️ Try the live demo here 👉 huggingface.co/spaces/wusize/… Supports multimodal inputs and text-to-image generation!

@LucasJinReal @EIFY @arankomatsuzaki They used (shown in Figure3b). It seems they are distilling CLIP’ knowledge to the LLM.

Unveiling Encoder-Free Vision-Language Models Achieves smaller performance-compute gap between encoder-based VLM and decoder-only VLM arxiv.org/abs/2406.11832

🌟Our paper proposes a clever and novel approach to Open-Vocabulary Object Detection. Come and check it in this afternoon #CVPR2023 ! ​ 📍 Location: West Exhibit Hall #276 ⏰ Time: 04: 30 PM - 06:00 PM Paper: arxiv.org/abs/2302.13996