Large Language Models and Knowledge Graphs: Opportunities and Challenges

Large Language Models (LLMs) have taken Knowledge Representation -- and the world -- by storm. This inflection point marks a shift from explicit knowledge representation to a renewed focus on the hybrid representation of both explicit knowledge and parametric knowledge. In this position paper, we will discuss some of the common debate points within the community on LLMs (parametric knowledge) and Knowledge Graphs (explicit knowledge) and speculate on opportunities and visions that the renewed focus brings, as well as related research topics and challenges.

• Publication year

  2023

• Venue

  TGDK

• Publication date

  2023-08-11

• Fields of study

  Computer Science

• Identifiers
  DOI 10.48550/arXiv.2308.06374arXiv 2308.06374
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints

  2024cited by this paper
• DeepOnto: A Python Package for Ontology Engineering with Deep Learning

  2023cited by this paper
• LLaMA: Open and Efficient Foundation Language Models

  2023cited by this paper
• Inference with Reference: Lossless Acceleration of Large Language Models

  2023cited by this paper
• FastRAT: Fast and Efficient Cross-lingual Text-to-SQL Semantic Parsing

  2023cited by this paper
• Improving Sequential Model Editing with Fact Retrieval

  2023cited by this paper
• WDC Products: A Multi-Dimensional Entity Matching Benchmark

  2023cited by this paper
• Knowledge graph extension with a pre-trained language model via unified learning method

  2023cited by this paper
• From Large Language Models to Databases and Back: A Discussion on Research and Education

  2023cited by this paper
• Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning

  2023cited by this paper
• A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?

  2023cited by this paper
• Retrieval Augmented Generation with Rich Answer Encoding

  2023cited by this paper
• Give us the Facts: Enhancing Large Language Models With Knowledge Graphs for Fact-Aware Language Modeling

  2023cited by this paper
• Evaluating Language Models for Knowledge Base Completion

  2023influential reference
• A Trip Towards Fairness: Bias and De-Biasing in Large Language Models

  2023cited by this paper
• Language Model Analysis for Ontology Subsumption Inference

  2023cited by this paper
• Which Stereotypes Are Moderated and Under-Moderated in Search Engine Autocompletion?

  2023cited by this paper
• Transformer-based approach for automated context-aware IFC-regulation semantic information alignment

  2023cited by this paper
• Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness

  2023cited by this paper
• Decoding Prompt Syntax: Analysing its Impact on Knowledge Retrieval in Large Language Models

  2023cited by this paper
• Subsumption Prediction for E-Commerce Taxonomies

  2023cited by this paper
• BloombergGPT: A Large Language Model for Finance

  2023cited by this paper
• Reimagining Retrieval Augmented Language Models for Answering Queries

  2023cited by this paper
• Large Language Models

  2023cited by this paper
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning

  2023cited by this paper
• The Biases of Pre-Trained Language Models: An Empirical Study on Prompt-Based Sentiment Analysis and Emotion Detection

  2023cited by this paper
• COFFEE: A Contrastive Oracle-Free Framework for Event Extraction

  2023cited by this paper
• An empirical study of pre-trained language models in simple knowledge graph question answering

  2023cited by this paper
• A divide and conquer framework for Knowledge Editing

  2023cited by this paper
• InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction

  2023cited by this paper
• Unifying Large Language Models and Knowledge Graphs: A Roadmap

  2023cited by this paper
• Extraction of Validating Shapes from very large Knowledge Graphs

  2023cited by this paper
• Zero-Shot Information Extraction via Chatting with ChatGPT

  2023cited by this paper
• Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction

  2023cited by this paper
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

  2023cited by this paper
• Deep learning-based relation extraction and knowledge graph-based representation of construction safety requirements

  2023cited by this paper
• Toolformer: Language Models Can Teach Themselves to Use Tools

  2023cited by this paper
• LLMs for knowledge graph construction and reasoning: recent capabilities and future opportunities

  2023influential reference
• Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion

  2023cited by this paper
• Towards Foundation Models for Relational Databases [Vision Paper]

  2023cited by this paper
• Generalizing to Unseen Elements: A Survey on Knowledge Extrapolation for Knowledge Graphs

  2023cited by this paper
• KILM: Knowledge Injection into Encoder-Decoder Language Models

  2023cited by this paper
• SHACTOR: Improving the Quality of Large-Scale Knowledge Graphs with Validating Shapes

  2023cited by this paper
• Does Synthetic Data Generation of LLMs Help Clinical Text Mining?

  2023cited by this paper
• Column Type Annotation using ChatGPT

  2023cited by this paper
• Complex Logical Reasoning over Knowledge Graphs using Large Language Models

  2023cited by this paper
• Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction

  2023cited by this paper
• Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective

  2022cited by this paper
• Language Models Are Poor Learners of Directional Inference

  2022cited by this paper
• A Review on Language Models as Knowledge Bases

  2022cited by this paper
• Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning

  2022cited by this paper
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

  2022cited by this paper
• The Lifecycle of “Facts”: A Survey of Social Bias in Knowledge Graphs

  2022cited by this paper
• SHACL and ShEx in the Wild: A Community Survey on Validating Shapes Generation and Adoption

  2022cited by this paper
• Shortcut Learning of Large Language Models in Natural Language Understanding

  2022cited by this paper
• Task-specific Pre-training and Prompt Decomposition for Knowledge Graph Population with Language Models

  2022cited by this paper
• PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts

  2022cited by this paper
• Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Prompts

  2022cited by this paper
• Endowing Language Models with Multimodal Knowledge Graph Representations

  2022cited by this paper
• Language Model Guided Knowledge Graph Embeddings

  2022cited by this paper
• LP-BERT: Multi-task Pre-training Knowledge Graph BERT for Link Prediction

  2022cited by this paper
• Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions

  2022cited by this paper
• Memory-Based Model Editing at Scale

  2022cited by this paper
• SKILL: Structured Knowledge Infusion for Large Language Models

  2022cited by this paper
• Technical Perspective of TURL

  2022cited by this paper
• A large language model for electronic health records

  2022cited by this paper
• SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models

  2022influential reference
• MADLINK: Attentive multihop and entity descriptions for link prediction in knowledge graphs

  2022cited by this paper
• Instruction Induction: From Few Examples to Natural Language Task Descriptions

  2022cited by this paper
• MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid

  2022cited by this paper
• Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models

  2022cited by this paper
• MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning

  2022cited by this paper
• From tabular data to knowledge graphs: A survey of semantic table interpretation tasks and methods

  2022cited by this paper
• Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model

  2022cited by this paper
• Challenge Overview

  2022cited by this paper
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

  2022cited by this paper
• Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation

  2022cited by this paper
• RARR: Researching and Revising What Language Models Say, Using Language Models

  2022cited by this paper
• Contextual semantic embeddings for ontology subsumption prediction

  2022cited by this paper
• Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching

  2022cited by this paper
• Chain of Thought Prompting Elicits Reasoning in Large Language Models

  2022cited by this paper
• Integrating Knowledge Graph embedding and pretrained Language Models in Hypercomplex Spaces

  2022cited by this paper
• KAMEL: Knowledge Analysis with Multitoken Entities in Language Models

  2022cited by this paper
• Large Language Models Are Human-Level Prompt Engineers

  2022cited by this paper
• KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models

  2022cited by this paper
• Extracting Cultural Commonsense Knowledge at Scale

  2022cited by this paper
• Improving Language Model Predictions via Prompts Enriched with Knowledge Graphs

  2022cited by this paper
• Towards Reasoning in Large Language Models: A Survey

  2022cited by this paper
• Understanding Gender Bias in Knowledge Base Embeddings

  2022cited by this paper
• Training language models to follow instructions with human feedback

  2022cited by this paper
• Prompting as Probing: Using Language Models for Knowledge Base Construction

  2022cited by this paper
• The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation

  2022cited by this paper
• Knowledge Graphs and their Role in the Knowledge Engineering of the 21st Century (Dagstuhl Seminar 22372)

  2022cited by this paper
• BERTMap: A BERT-based Ontology Alignment System

  2021cited by this paper
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction

  2021cited by this paper
• Large pre-trained language models contain human-like biases of what is right and wrong to do

  2021cited by this paper
• Prix-LM: Pretraining for Multilingual Knowledge Base Construction

  2021cited by this paper
• Contextual Language Models for Knowledge Graph Completion

  2021cited by this paper
• Predicting Document Coverage for Relation Extraction

  2021cited by this paper
• Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark

  2021cited by this paper

• Automated carbon-aware assessment of openBIM-based ductwork design using knowledge graph–augmented LLM multi-agent framework

  2026cites this paper
• Combining Structural and Textual Knowledge for Knowledge Graph Link Prediction via Large Language Models

  2026cites this paper
• A knowledge-formalization and vision-language-model system for fire emergency decision support

  2026cites this paper
• Diagnosing Structural Failures in LLM-Based Evidence Extraction for Meta-Analysis

  2026cites this paper
• Unlocking the Potential of Large Language Models in Education: Factors Influencing Adoption by Instructional Designers and Academics

  2026influential citation
• FAIR GraphRAG: A Retrieval-Augmented Generation Approach for Semantic Data Analysis

  2025cites this paper
• A Roadmap to Graph Analytics

  2025cites this paper
• Survey on terminology extraction from texts

  2025cites this paper
• K-ON: Stacking Knowledge On the Head Layer of Large Language Model

  2025cites this paper
• “Conversing” With Qualitative Data: Enhancing Qualitative Research Through Large Language Models (LLMs)

  2025cites this paper
• SIGFRID: Unsupervised, Platform-Agnostic Interference Detection in IoT Automation Rules

  2025cites this paper
• The Interpretable Reasoning and Intelligent Decision-Making Based on Event Knowledge Graph With LLMs in Fault Diagnosis Scenarios

  2025cites this paper
• Knowledge Graphs for Multi-modal Learning: Survey and Perspective

  2025cites this paper
• Taxonomy-Aware Evaluation of Vision–Language Models

  2025cites this paper
• KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs

  2025cites this paper
• Leveraging Large Language Models for Ontology Requirements Engineering

  2025cites this paper
• Graph-Based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey

  2025cites this paper
• Multi-modal Knowledge Graph Generation with Semantics-enriched Prompts

  2025cites this paper
• Knowledge Graphs and Their Reciprocal Relationship with Large Language Models

  2025influential citation
• Scalable Knowledge Graph Construction from Unstructured Text: A Case Study on Artisanal and Small-Scale Gold Mining

  2025cites this paper
• Autofocus Retrieval: An Effective Pipeline for Multi-Hop Question Answering With Semi-Structured Knowledge

  2025cites this paper
• Benchmarking methods for classifying space functions and access elements in multi-unit apartment buildings

  2025cites this paper
• MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations

  2025cites this paper
• The Role of Knowledge Graphs on Responsible Artificial Intelligence Realization: Research Opportunities and Challenges

  2025cites this paper
• Pretrained LLMs Learn Multiple Types of Uncertainty

  2025cites this paper
• KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models

  2025cites this paper
• Hallucinate, Ground, Repeat: A Framework for Generalized Visual Relationship Detection

  2025cites this paper
• JOINT FEATURES-BASED KNOWLEDGE GRAPH COMPLETION

  2025cites this paper
• Gender bias in machine learning: insights from official labour statistics and textual analysis

  2025cites this paper
• KERAP: A Knowledge-Enhanced Reasoning Approach for Accurate Zero-shot Diagnosis Prediction Using Multi-agent LLMs

  2025influential citation
• Practices, opportunities and challenges in the fusion of knowledge graphs and large language models

  2025cites this paper
• Simulation Deduction Method for Operation Order Instructions based on Anti-misoperation Knowledge Graph

  2025cites this paper
• Knowledge Graphs vs. Large Language Models: Competitors or Partners in Supporting Virtual Museums

  2025cites this paper
• Facilitating Design for Additive Manufacturing with KG-based Retrieval-Augmented Generation

  2025cites this paper
• Large Language Models and Knowledge Graphs: A State-of-the-Art Exploration

  2025influential citation
• Synergizing a knowledge graph and large language model for relay catalysis pathway recommendation

  2025cites this paper
• Reconstructing Judicial Digital Forensic Evidence Graphs from Legal Documents Using Large Language Models

  2025cites this paper
• A Hybrid, Neuro-symbolic Approach for Scholarly Knowledge Organization

  2025cites this paper
• Toward Reliable Knowledge Graphs in Virtual Product Development: Data Strategies and Reproducibility Assessment

  2025cites this paper
• CLAUSE: Agentic Neuro-Symbolic Knowledge Graph Reasoning via Dynamic Learnable Context Engineering

  2025cites this paper
• From Fragmentation to Cohesion: An LLM-Based Iterative Approach to Ontology and Knowledge Graph Refinement

  2025cites this paper
• The New Quant: A Survey of Large Language Models in Financial Prediction and Trading

  2025cites this paper
• Prompting large language models based on semantic schema for text-to-Cypher transformation towards domain Q&A

  2025cites this paper
• Textbook2KG: Decoding Knowledge Graph from Very Long Textual Content via Prompt Engineering

  2025cites this paper
• Towards a FAIR knowledge graph-based conversational agent for exploring agroforestry research

  2025cites this paper
• Right Answer at the Right Time - Temporal Retrieval-Augmented Generation via Graph Summarization

  2025cites this paper
• Benchmarking Fairness-aware Graph Neural Networks in Knowledge Graphs

  2025cites this paper
• Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challenges

  2025cites this paper
• CMOMgen: Complex Multi-Ontology Alignment via Pattern-Guided In-Context Learning

  2025cites this paper
• Towards Explainable Automated Knowledge Engineering With Human-in-the-Loop

  2025cites this paper
• From Logs to Knowledge: LLM-Powered Dynamic Knowledge Graphs for Real-Time Cloud Observability

  2025cites this paper
• Introducing GPTKB to the Semantic Web

  2025cites this paper
• Distribution-Guided Extraction for Cross-Platform Educational Knowledge Graph Construction

  2025cites this paper
• Enhancing data discovery with contextual pre-filtering

  2025cites this paper
• RehabKD With LLM-KG-LDACX: Orchestrating AI Models for Rehabilitation Knowledge Discovery

  2025cites this paper
• Hybrid Knowledge Graph Integration in Large Language Models for Explainable Contextual Understanding and Improved Accuracy

  2025cites this paper
• Exploring the Usage of Knowledge Graphs in Identifying Human and LLM-Generated Fake Reviews

  2025cites this paper
• Mind the Context: Enriching Knowledge Graphs with Rules and Mappings

  2025cites this paper
• I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token

  2024cites this paper
• An LLM supported approach to ontology and knowledge graph construction

  2024cites this paper
• BIFROST: A Future Graph Database Runtime

  2024cites this paper
• Numerical Literals in Link Prediction: A Critical Examination of Models and Datasets

  2024cites this paper
• Making knowledge graphs work for smart manufacturing: Research topics, applications and prospects

  2024influential citation
• Generating Troubleshooting Trees for Industrial Equipment using Large Language Models (LLM)

  2024cites this paper
• ASGM-KG: Unveiling Alluvial Gold Mining Through Knowledge Graphs

  2024cites this paper
• Knowledge prompting: How knowledge engineers use generative AI

  2024cites this paper
• Harnessing the Power of Semi-Structured Knowledge and LLMs with Triplet-Based Prefiltering for Question Answering

  2024cites this paper
• OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System

  2024cites this paper
• A patient-centric knowledge graph approach to prioritize mutants for selective anti-cancer targeting

  2024cites this paper
• EXPLORING LARGE LANGUAGE MODELS FOR ONTOLOGY LEARNING

  2024cites this paper
• MKGL: Mastery of a Three-Word Language

  2024cites this paper
• On the Influence of Shape, Texture and Color for Learning Semantic Segmentation

  2024cites this paper
• Resilience in Knowledge Graph Embeddings

  2024cites this paper
• Can Large Language Models Act as Symbolic Reasoners?

  2024cites this paper
• The Deep Integration of Knowledge Graphs and Large Language Models: Advancements, Challenges, and Future Directions

  2024cites this paper
• Ontology Population using LLMs

  2024cites this paper
• Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective

  2024cites this paper
• Human Evaluation of Procedural Knowledge Graph Extraction from Text with Large Language Models

  2024cites this paper
• Spectral Clustering-Guided News Environments Perception for Fake News Detection

  2024cites this paper
• A Pluggable Common Sense-Enhanced Framework for Knowledge Graph Completion

  2024cites this paper
• KGLM-QA: A Novel Approach for Knowledge Graph-Enhanced Large Language Models for Question Answering

  2024cites this paper
• A Comprehensive Study of Knowledge Editing for Large Language Models

  2024cites this paper
• Prompt-Time Symbolic Knowledge Capture with Large Language Models

  2024cites this paper
• A Survey on Extractive Knowledge Graph Summarization: Applications, Approaches, Evaluation, and Future Directions

  2024cites this paper
• Large multimodal agents: a survey

  2024cites this paper
• On the Challenges and Opportunities in Generative AI

  2024cites this paper
• From human experts to machines: An LLM supported approach to ontology and knowledge graph construction

  2024influential citation
• Large language models as oracles for instantiating ontologies with domain-specific knowledge

  2024cites this paper
• Building A Knowledge Graph to Enrich ChatGPT Responses in Manufacturing Service Discovery

  2024cites this paper
• Augmented non-hallucinating large language models as medical information curators

  2024cites this paper
• Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts

  2024cites this paper
• AIREG: Enhanced Educational Recommender System with Large Language Models and Knowledge Graphs

  2024cites this paper
• Assessing the Evolution of LLM Capabilities for Knowledge Graph Engineering in 2023

  2024cites this paper
• The Role of Generative AI in Competency Question Retrofitting

  2024cites this paper
• Large language models: Expectations for semantics-driven systems engineering

  2024cites this paper
• Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning

  2024cites this paper
• Uncertainty Management in the Construction of Knowledge Graphs: a Survey

  2024influential citation
• EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs

  2024cites this paper
• Research Trends for the Interplay between Large Language Models and Knowledge Graphs

  2024cites this paper
• TRUE-UIE: Two Universal Relations Unify Information Extraction Tasks

  2024cites this paper