Multi-label node classification in heterogeneous networks using graph convolutional networks

Articles in Press

Document Type : Research Paper

Authors

1 Department of Computer Engineering, Kordestan University, Sanandej, Iran

2 Department of Computer Engineering, Lorestan University, Khoramabad, Iran

Abstract

This paper explores graph embedding techniques for effectively analyzing large, heterogeneous graphs with complex and noisy patterns. Graphs represent data through nodes (entities) and edges (relationships), and when dealing with large-scale data, effective search methods are crucial. Graph embedding helps evaluate node significance and transforms data into latent space representations. It also addresses challenges like handling multi-label data in heterogeneous networks, where nodes may have multiple labels describing complex concepts. Traditional methods struggle with such multi-label scenarios and fail to capture label dependencies. The paper introduces a Graph Neural Network (GCN)-based node embedding method, which extends traditional neural networks to graph data. GCNs allow the extraction of local features from nodes and their neighbors, making them useful for heterogeneous networks. By integrating label information into the embedding process, the method improves relationships between labels. The proposed approach transforms neighboring labels into continuous vectors, structured into a matrix for learning. This enhances the overall network embedding. The method outperforms previous techniques, demonstrating improved performance on real-world datasets, such as a 2.4% improvement on the IMDB dataset and 9.3% on the DBLP dataset. The paper discusses graph embedding techniques in the first section and explores the potential of multi-label embedding in non-uniform graphs, suggesting future research directions in the final section. The article's code link on GitHub can also be found at the following: https://github.com/frshkara/EGSA.

Keywords

Main Subjects

References

[1] Zhang, M.-L. and Z.-H. Zhou, A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, 2013. 26(8): p. 1819-1837, doi://doi.org/10.1109/TKDE.2013.39.
[2] Read, J., et al., Classi er chains for multi-label classi cation. Machine learning, 2011. 85(3): p. 333-359, doi://doi.org/10.1007/s10994-011-5256-5.
[3] V. Chauhan and A. Tiwari, \Randomized neural networks for multilabel classi cation," Appl. Soft Comput., vol. 115, p. 108184, Jan. 2022, doi://doi.org/10.1016/j.asoc.2021.108184.
[4] M.A. Tahir, J. Kittler, and A. Bouridane, \Multilabel classi cation using heterogeneous ensemble of multi-label classi ers," Pattern Recognit. Lett., vol. 33, no. 5, pp. 513-523, Apr. 2012, doi://doi.org/10.1016/j.patrec.2011.10.019.
[5] Tsitsulin, A., et al., Graph clustering with graph neural networks. arXiv preprint arXiv:2006.16904, 2020, doi://doi.org/10.48550/arXiv.2006.16904.
[6] Kipf, T.N. and M. Welling, Semi-supervised classi cation with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016, doi://doi.org/10.48550/arXiv.1609.02907.
[7] You, J., Z. Ying, and J. Leskovec, Design space for graph neural networks. Advances in Neural Information Processing Systems, 2020. 33, doi://doi.org/10.48550/arXiv.2011.08843.
[8] Gibaja, E. and S. Ventura, A tutorial on multilabel learning. ACM Computing Surveys (CSUR), 2015. 47(3): p. 1-38, doi://doi.org/10.1145/2716262.
[9] Wu, Z., et al., A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 2020. 32(1): p. 4-24, doi://doi.org/10.1109/TNNLS.2020.2978386.
[10] Kumar, V., et al., Multi-label classi cation using hierarchical embedding. Expert Systems with Applications, 2018. 91: p. 263-269, doi://doi.org/10.1016/j.eswa.2017.09.020.
[11] Vembu, S. and T. Gartner, Label ranking algorithms: A survey, in Preference learning. 2010, Springer. p. 45-64, doi://doi.org/10.1007/978-3-642-14125-6-3.
[12] Liu, S., et al., Query2Label: A Simple Transformer Way to Multi-Label Classi cation. arXiv preprint arXiv:2107.10834, 2021, doi://doi.org/10.48550/arXiv.2107.10834.
[13] Hamilton, W.L., R. Ying, and J. Leskovec, Representation learning on graphs: Methods and applications. arXiv preprint arXiv:1709.05584, 2017, doi://doi.org/10.48550/arXiv.1709.05584.
[14] Mikolov, T., et al. Distributed representations of words and phrases and their compositionality. in Advances in neural information processing systems. 2013, doi://doi.org/10.48550/arXiv.1310.4546.
[15] Perozzi, B., R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014, doi://doi.org/10.1145/2623330.2623732.
[16] Huang, X., J. Li, and X. Hu. Label informed attributed network embedding. in Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 2017, doi://doi.org/10.1145/3018661.3018667.
[17] Shi, M., Y. Tang, and X. Zhu, MLNE: Multi-label network embedding. IEEE transactions on neural networks and learning systems, 2019. 31(9): p. 3682-3695, doi://doi.org/10.1109/tnnls.2019.2945869.
[18] Guo, Z., et al., A two-level topic model towards knowledge discovery from citation networks. IEEE Transactions on Knowledge and Data Engineering, 2013. 26(4): p. 780-794, doi://doi.org/10.1109/TKDE.2013.56.
[19] Zhang, L., et al., A modi ed stochastic neighbor embedding for multi-feature dimension reduction of remote sensing images. ISPRS journal of photogrammetry and remote sensing, 2013. 83: p. 30-39, doi://doi.org/10.5194/isprsannals-I-3-395-2012.
[20] Szymanski, P., T. Kajdanowicz, and N. Chawla, LNEMLC: Label network embeddings for multi-label classi cation. arXiv preprint arXiv:1812.02956, 2018, doi://doi.org/10.48550/arXiv.1812.02956.
[21] Sun, X., et al., Label embedding network: Learning label representation for soft training of deep networks. arXiv preprint arXiv:1710.10393, 2017, doi://doi.org/10.48550/arXiv.1710.10393.
[22] Cai, H., V.W. Zheng, and K.C.-C. Chang, A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 2018. 30(9): p. 1616-1637, doi://doi.org/10.48550/arXiv.1709.07604.
[23] R. Rastogi and S. Mortaza, \Multi-label classi cation with Missing Labels using Label Correlation and Robust Structural Learning," Knowl.-Based Syst., vol. 229, p. 107336, Oct. 2021, doi://doi.org/10.1016/j.knosys.2021.107336.
[24] I. Ullah, M. Manzo, M. Shah, and M. G. Madden, \Graph convolutional networks: analysis, improvements and results," Appl. Intell., vol. 52, no. 8, pp. 9033-9044, Jun. 2022, doi://doi.org/10.1007/s10489-021-02973-4.
[25] Meihao Chen, Zhuoru Lin, Kyunghyun Cho. Graph Convolutional Networks for Classi cation with a Structured Label Space. arXiv preprint arXiv: 1710.04908, 2017, doi://doi.org/10.48550/arXiv.1710.04908.
[26] Z. Li, Y. Liu, Z. Zhang, S. Pan, J. Gao, and J. Bu, \Semi-supervised classication with graph convolutional networks,". arXiv preprint arXiv:1609.02907, 2016, doi://doi.org/10.48550/arXiv.1609.02907.
[27] Y. Wu, Y. Song, H. Huang, F. Ye, X. Xie, and H. Jin, \Enhancing Graph Neural Networks via auxiliary training for semi-supervised node classi cation," Knowl.-Based Syst., vol. 220, p. 106884, May 2021, doi://doi.org/10.1016/j.knosys.2021.106884.
[28] Thomas N Kipf and Max Welling. Semi-supervised classi cation with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016, doi://doi.org/10.48550/arXiv.1609.02907.
[29] Shi, M., et al., Multi-Label Graph Convolutional Network Representation Learning. IEEE Transactions on Big Data, 2020, doi://doi.org/10.1109/TBDATA.2020.3019478.
[30] Y. Du, W. Guo, J. Liu, and C. Yao, \Classi cation by multi-semantic meta path and active weight learning in heterogeneous information networks," Expert Syst. Appl., vol. 123, pp. 227-236, Jun. 2019, doi://doi.org/10.1016/j.eswa.2019.01.044.
[31] Y. Xiao, P. Quan, M. Lei, and L. Niu, \Latent neighborhood-based heterogeneous graph representation," Neural Netw., vol. 154, pp. 413-424, Oct. 2022, doi://doi.org/10.1016/j.neunet.2022.07.028.
[32] L. Zong, X. Zhang, H. Yu, Q. Zhao, and F. Ding, \Local linear neighbor reconstruction for multi-view data," Pattern Recognit. Lett., vol. 84, pp. 56-62, Dec. 2016, doi://doi.org/10.1016/j.patrec.2016.08.002.
[33] N. Tong, Y. Tang, B. Chen, and L. Xiong, \Representation learning using Attention Network and CNN for Heterogeneous networks," Expert Syst. Appl., vol. 185, p. 115628, Dec. 2021, doi://doi.org/10.1016/j.eswa.2021.115628.
[34] T. Zhao, N. T. Dong, A. Hanjalic, and M. Khosla, \Multi-label Node Classi cation On Graph-Structured Data," 2023, doi: 10.48550/ARXIV.2304.10398.
[35] Z. Qin, H. Chen, Y. Mi, C. Luo, S.-J. Horng, and T. Li, \Multi-label Feature selection with adaptive graph learning and label information enhancement,"Knowledge-Based Systems, vol. 285, p. 111363, Feb. 2024, doi: 10.1016/j.knosys.2023.111363.
Journal of Mahani Mathematical Research

Articles in Press, Accepted Manuscript
Available Online from 02 January 2025

Files

History

  • Receive Date: 27 May 2024
  • Revise Date: 28 September 2024
  • Accept Date: 01 November 2024

Share

How to cite

Statistics