Recent innovations in large language model (LLM) design have led to rapid advancements in few-shot learning and reasoning capabilities. However, despite their progress, LLMs still face limitations when dealing with complex real-world contexts involving massive amounts of interconnected knowledge.
To address this challenge, a promising approach has emerged in retrieval augmented generation (RAG) systems. RAG combines the adaptive learning strengths of LLMs with scalable retrieval from external knowledge sources like knowledge graphs (KGs). Rather than attempting to encode all information within the model statically, RAG allows querying necessary context from indexed knowledge graphs on the fly as needed.
However, effectively orchestrating reasoning and retrieval across interconnected knowledge sources brings its own challenges. Naive approaches that simply retrieve and concatenate information in discrete steps often fail to fully capture the nuances within dense knowledge graphs. The interconnected nature of concepts means that vital contextual details can be missed if not analyzed in relation to one another.
Recently, an intriguing framework named LLM Compiler has demonstrated early successes in optimizing orchestration of multiple function calls in LLMs by automatically handling dependencies and allowing parallel execution.
In this article, we explore the potential of applying LLM Compiler techniques more broadly to knowledge graph retrieval and reasoning. We already did a working prototype before the paper released :