Action-conditioned video data improves predictability

التفاصيل البيبلوغرافية
العنوان: Action-conditioned video data improves predictability
المؤلفون: Sarkar, Meenakshi, Ghose, Debasish
سنة النشر: 2024
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Computer Vision and Pattern Recognition
الوصف: Long-term video generation and prediction remain challenging tasks in computer vision, particularly in partially observable scenarios where cameras are mounted on moving platforms. The interaction between observed image frames and the motion of the recording agent introduces additional complexities. To address these issues, we introduce the Action-Conditioned Video Generation (ACVG) framework, a novel approach that investigates the relationship between actions and generated image frames through a deep dual Generator-Actor architecture. ACVG generates video sequences conditioned on the actions of robots, enabling exploration and analysis of how vision and action mutually influence one another in dynamic environments. We evaluate the framework's effectiveness on an indoor robot motion dataset which consists of sequences of image frames along with the sequences of actions taken by the robotic agent, conducting a comprehensive empirical study comparing ACVG to other state-of-the-art frameworks along with a detailed ablation study.
نوع الوثيقة: Working Paper
الوصول الحر: http://arxiv.org/abs/2404.05439Test
رقم الانضمام: edsarx.2404.05439
قاعدة البيانات: arXiv