Skip to content

Latest commit

 

History

History
34 lines (29 loc) · 3.94 KB

MMKG.md

File metadata and controls

34 lines (29 loc) · 3.94 KB

Multi-Modal Knowledge Graph (MMKG)

Surveys and Paper Summary

  1. Multi-Modal Knowledge Graph Construction and Application: A Survey (Arxiv 2022, from Prof Yanghua Xiao) [Paper] [Notes]'
  2. What Is a Multi-Modal Knowledge Graph: A Survey (Big Data Research, 2023) [Paper]
  3. A Survey on Multimodal Knowledge Graphs: Construction, Completion and Applications (Mathematics 2023) [Paper]
  4. Awesome-Multimodal-Knowledge-Graph [GitHub]
  5. A collection of Multimoal KG Resources [GitHub]
  6. Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey (Arxiv, Feb 2024) [Paper] [GitHub (KG-MM-Survey)]

General Papers

  1. MMEKG: Multi-modal Event Knowledge Graph towards Universal Representation across Modalities (ACL 2022) [Paper]
  2. Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency (ICMR 20202) [Paper]
  • The authors work on two datasets TamperedNews & News400v (updated in IJMIR 2021) [Link]
  1. Toward a General Framework for Multimodal Big Data Analysis (Big Data 2022) [Paper]
  • Multiple data sources are first integrated into a unified knowledge graph (KG). Different modalities of data are addressed by specifying ad hoc views on the KG and producing a rewriting of the graph containing merely the data to be processed. Graph traversal and rule extraction are this way boosted.
  1. Multimodal Reasoning with Multimodal Knowledge Graphs (ACL 2024) [Paper] [Notes]

Multimodal entity linking (MEL)

  1. Multi-Grained Multimodal Interaction Network for Entity Linking (KDD 2023) [Paper]
  2. A Multimodal Entity Linking Approach Incorporating Topic Concepts (CISAI 2022) [Paper]
  3. Multimodal entity linking for tweets (HAL Open Research 2020) [Paper]
  4. Visual Entity Linking via Multi-modal Learning (Data Intelligence 2022) [Paper]
  5. Multimodal Entity Tagging with Multimodal Knowledge Base (Arxiv 2022) [Paper]
  6. Generative Multimodal Entity Linking (Arxiv 2023) [Paper] 🔥
  7. Integrating Large Language Models in Multimodal Entity Linking: A Novel Two-Level Reflection Framework (submitted to ACL 2023, open review) [Paper] 🔥

Benchmarks, Datasets and Evaluation

  1. (MEL Dataset) WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types (ACL 2022) [Paper]
  2. (MEL Dataset) Multimodal Entity Linking: A New Dataset and A Baseline (MM 2021) [Paper]
  3. (MEL Dataset) Building a multimodal entity linking dataset from tweets (HAL Open Research 2020) [Paper]

MMKG for LLM

  1. Endowing Language Models with Multimodal Knowledge Graph Representations (Arxiv 2022) [Paper]