Robust Mask-Guided Matting: Managing Noisy Inputs And Object Versatility

Robust Mask-Guided Matting: Managing Noisy Inputs and Object Versatility | HackerNoon

Last updated: 2025/12/20 at 6:29 PM

News Room Published 20 December 2025

Table of Links

Abstract and 1. Introduction

Related Works
MaGGIe

3.1. Efficient Masked Guided Instance Matting

3.2. Feature-Matte Temporal Consistency
Instance Matting Datasets

4.1. Image Instance Matting and 4.2. Video Instance Matting
Experiments

5.1. Pre-training on image data

5.2. Training on video data
Discussion and References

Supplementary Material

Architecture details
Image matting

8.1. Dataset generation and preparation

8.2. Training details

8.3. Quantitative details

8.4. More qualitative results on natural images
Video matting

9.1. Dataset generation

9.2. Training details

9.3. Quantitative details

9.4. More qualitative results

8.4. More qualitative results on natural images

Fig. 13 showcases our model’s performance in challenging scenarios, particularly in accurately rendering hair regions. Our framework consistently outperforms MGM⋆ in detail preservation, especially in complex instance interactions. In comparison with InstMatt, our model exhibits superior instance separation and detail accuracy in ambiguous regions.

Fig. 14 and Fig. 15 illustrate the performance of our model and previous works in extreme cases involving multiple instances. While MGM⋆ struggles with noise and accuracy in dense instance scenarios, our model maintains high precision. InstMatt, without additional training data, shows limitations in these complex settings.

The robustness of our mask-guided approach is further demonstrated in Fig. 16. Here, we highlight the challenges faced by MGM variants and SparseMat in predicting missing parts in mask inputs, which our model addresses. However, it is important to note that our model is not designed as a human instance segmentation network. As shown in Fig. 17, our framework adheres to the input guidance, ensuring precise alpha matte prediction even with multiple instances in the same mask.

Lastly, Fig. 12 and Fig. 11 emphasize our model’s generalization capabilities. The model accurately extracts both human subjects and other objects from backgrounds, showcasing its versatility across various scenarios and object types.

All examples are Internet images without ground-truth and the mask from r101fpn400e are used as the guidance.

Figure 14. Our frameworks precisely separate instances in an extreme case with many instances. While MGM often causes the overlapping between instances and MGM⋆ contains noises, ours produces on-par results with InstMatt trained on the external dataset. Red arrow indicates the errors. (Best viewed in color and digital zoom).

Figure 15. Our frameworks precisely separate instances in a single pass. The proposed solution shows comparable results with InstMatt and MGM without running the prediction/refinement five times. Red arrow indicates the errors. (Best viewed in color and digital zoom).

Figure 16. Unlike MGM and SparseMat, our model is robust to the input guidance mask. With the attention head, our model produces more stable results to mask inputs without complex refinement between instances like InstMatt. Red arrow indicates the errors. (Best viewed in color and digital zoom).

Figure 17. Our solution works correctly with multi-instance mask guidances. When multiple instances exist in one guidance mask, we still produce the correct union alpha matte for those instances. Red arrow indicates the errors or the zoom-in region in red box. (Best viewed in color and digital zoom).

Table 12. Details of quantitative results on HIM2K+M-HIM2K (Extension of Table 5). Gray indicates the public weight without retraining.

Table 12. Details of quantitative results on HIM2K+M-HIM2K (Extension of Table 5). Gray indicates the public weight without retraining. (Continued)

Table 13. The effectiveness of proposed temporal consistency modules on V-HIM60 (Extension of Table 6). The combination of bi-directional Conv-GRU and forward-backward fusion achieves the best overall performance on three test sets. Bold highlights the best for each level.