Knowledge Graph as Guardrails: Achieving Conceptually Sound LLM Complaint Classification Without Fine-Tuning

Sudjianto, A.

Large Language Models (LLMs) have shown remarkable capabilities in various classification tasks, but their deployment in high-stakes environments like banking introduces significant risks, such as when dealing with subtly expressed complaints. This tutorial paper addresses the critical intersection of two challenges: achieving conceptual soundness in LLM applications without fine-tuning, as required by banking regulations, and accurately identifying complaint indicators beneath subtle language. By implementing knowledge graphs as explicit decision frameworks, we present a methodology for making pre-trained LLMs conceptually sound and trustworthy for complaint classification. Our approach emphasizes structured decomposition of decisions, explicit rule implementation, component-level validation, and transparent decision tracing to ensure alignment with regulatory expectations, all without requiring domain-specific fine-tuning of the underlying models.

Interested in reading the full article? Please visit the link below: