pipeline-overview

Pipeline Overview

This section aims to provide provide a comprehensive overview of the design and implementation of the pipeline. It include the following components:

Note: in this version of documentation, Stage 3: Entity Linking contains the mechanism of LLM double check before grouping Mentions into the same Entities. While it can increase the efficacy, it significantly reduce the efficiency of the pipeline as this step can lead to quadratic complexity. Therefore, this step is disable when the paper is submitted.

• Design of the open local KGC approach:

  • Demo
  • Comparison to Traditional KGC
  • Comparison to Other KGC
  • SWOT Analysis

• Overview of the Implementation and pipeline IO:

  • Interface
  • Pipeline Structure
  • Execution Diagram
  • Design Choice

• Stage 1: Mention Extraction

  • Algorithm
  • Prompts
    • Prompt 1: Named Entity Extraction
    • Prompt 1: Named Entity + General Concept Extraction
    • Prompt 1: Entity Extraction
  • Assumptions
  • Limitations
  • Complexity
  • Design Choices

• Stage 2: Entity Linking

  • Algorithm
  • Prompts
    • Prompt 2.1: LLM Background Knowledge Check
    • Prompt 2.2.1: Description Generation with Full Context
    • Prompt 2.2.2: Description Generation with Limited Context
    • Prompt 2.3: LLM Double Checking (Disabled)
  • Assumptions
  • Limitations
  • Complexity
  • Design Choices

• Stage 3: Local Relation Extraction

  • Algorithm
  • Prompts
    • Prompt 3: Local Relation Extraction
  • Assumptions
  • Limitations
  • Complexity
  • Design Choices

• Stage 4: Global Relation Extraction

  • Algorithm
  • Prompts
    • Prompt 4.1: Paper Summarisation
    • Prompt 4.2: Extracting A Predicate between Two Entities
    • Prompt 4.2+: Determining the Subject of the Relation
  • Assumptions
  • Limitations
  • Complexity
  • Design Choices

• Stage 5: Taxonomy Generation and Predication Resolution

  • Algorithm
  • Prompts
    • Prompt 5.1: Entity Type Description Generation
    • Prompt 5.2: Predicate Description Generation
  • Assumptions
  • Limitations
  • Complexity
  • Design Choices