Abstract

Detection transformers have been applied to human-object interaction (HOI) detection, enhancing the localization and recognition of human-action-object triplets in images. Despite remarkable progress, this study identifies a critical issue-"Toxic Siblings" bias-which hinders the interaction decoder's learning, as numerous similar yet distinct HOI triplets interfere with and even compete against each other both input side and output side to the interaction decoder. This bias arises from high confusion among sibling triplets/categories, where increased similarity paradoxically reduces precision, as one's gain comes at the expense of its toxic sibling's decline. To address this, we propose two novel debiasing learning objectives-"contrastive-then-calibration" and "merge-then-split"-targeting the input and output perspectives, respectively. The former samples sibling-like incorrect HOI triplets and reconstructs them into correct ones, guided by strong positional priors. The latter first learns shared features among sibling categories to distinguish them from other groups, then explicitly refines intra-group differentiation to preserve uniqueness. Experiments show that we significantly outperform both the baseline (+9.18% mAP on HICO-Det) and the state-of-the-art (+3.59% mAP) across various settings.

Framework

Experiment

Conclusion

In this work, we analyze key learning biases in human-object interaction (HOI) detection. To address these biases, we introduce two additional learning objectives. For input-level bias, we employ contrastive and calibration losses. To tackle output-level bias, we propose a merge-split learning strategy. Experimental results demonstrate that our approach significantly outperforms the baseline and state-of-the-art methods.

Acknowledgement

This work is supported by the National Natural Science Foundation of China (No.62206174).