DiscoverDaily Paper CastAffordance-Aware Object Insertion via Mask-Aware Dual Diffusion
Affordance-Aware Object Insertion via Mask-Aware Dual Diffusion

Affordance-Aware Object Insertion via Mask-Aware Dual Diffusion

Update: 2024-12-21
Share

Description

🤗 Upvotes: 13 | cs.CV



Authors:

Jixuan He, Wanhua Li, Ye Liu, Junsik Kim, Donglai Wei, Hanspeter Pfister



Title:

Affordance-Aware Object Insertion via Mask-Aware Dual Diffusion



Arxiv:

http://arxiv.org/abs/2412.14462v1



Abstract:

As a common image editing operation, image composition involves integrating foreground objects into background scenes. In this paper, we expand the application of the concept of Affordance from human-centered image composition tasks to a more general object-scene composition framework, addressing the complex interplay between foreground objects and background scenes. Following the principle of Affordance, we define the affordance-aware object insertion task, which aims to seamlessly insert any object into any scene with various position prompts. To address the limited data issue and incorporate this task, we constructed the SAM-FB dataset, which contains over 3 million examples across more than 3,000 object categories. Furthermore, we propose the Mask-Aware Dual Diffusion (MADD) model, which utilizes a dual-stream architecture to simultaneously denoise the RGB image and the insertion mask. By explicitly modeling the insertion mask in the diffusion process, MADD effectively facilitates the notion of affordance. Extensive experimental results show that our method outperforms the state-of-the-art methods and exhibits strong generalization performance on in-the-wild images. Please refer to our code on https://github.com/KaKituken/affordance-aware-any.

Comments 
In Channel
GUI Agents: A Survey

GUI Agents: A Survey

2024-12-2021:01

loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Affordance-Aware Object Insertion via Mask-Aware Dual Diffusion

Affordance-Aware Object Insertion via Mask-Aware Dual Diffusion

Jingwen Liang, Gengyu Wang