- 1次围观
Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field. In this paper, we present the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data. By instruction tuning on such generated data, we introduce LLaVA: Large Language and Vision Assistant, an end-to-end trained large multimodal model that connects a vision encoder and LLM for general-purpose visual and language understanding.Our early experiments show that LLaVA demonstrates impressive multimodel chat abilities, sometimes exhibiting the behaviors of multimodal GPT-4 on unseen images/instructions, and yields a 85.1% relative score compared with GPT-4 on a synthetic multimodal instruction-following dataset. When fine-tuned on Science QA, the synergy of LLaVA and GPT-4 achieves a new state-of-the-art accuracy of 92.53%. We make GPT-4 generated visual instruction tuning data, our model and code base publicly available.
来源出处
Visual Instruction Tuning
http://arxiv.org/abs/2304.08485
相关内容
发布日期
06/23/2024 - 17:52
发布日期
03/19/2024 - 09:13
发布日期
01/10/2022 - 19:31
发布日期
10/31/2021 - 01:16
发布日期
11/09/2024 - 19:46
发布日期
06/17/2022 - 10:21
发布日期
11/17/2024 - 19:48
发布日期
10/08/2023 - 23:02
发布日期
07/23/2023 - 21:46
发布日期
06/17/2022 - 10:21
发布日期
08/04/2020 - 01:35
发布日期
03/11/2025 - 20:51
发布日期
01/10/2022 - 19:31
发布日期
01/10/2022 - 19:31
发布日期
06/17/2022 - 10:21
发布日期
06/17/2022 - 10:21
发布日期
10/14/2023 - 23:10
发布日期
10/19/2024 - 19:37
发布日期
06/05/2024 - 17:45
发布日期
10/31/2021 - 01:12