FAIR Data Assessment Using LLMs: The Fair-Way

As part of modern research practices, the FAIR data principles have become essential for data discoverability, usability, and sharing. Existing implementations for automatically assessing FAIR adherence (FAIRness) often suffer from limited usability, inconsistent accuracy, and difficult-to-interpret results, as they require explicit rules to cover for specific FAIR assessment frameworks, which are not easy to generalize. This paper introduces Fair-Way, an open source tool that leverages Large Language Models (LLMs) to automate FAIRness assessment. Fair-Way applies a divide-and-conquer approach to decompose the assessment process into fine-grained tasks, as well as to split the metadata into manageable chunks. Evaluation demonstrates that Fair-Way achieves performance comparable to existing tools, while outperforming them in several key metrics. Moreover, Fair-Way generalizes across FAIR assessment indicators without requiring explicitly programmed logic and supports both structured and unstructured metadata in diverse formats. Finally, it enables user-defined, domain-specific tests, which are typically not supported by other systems. Overall, Fair-Way represents a scalable and flexible solution to accelerate FAIR data practices across research domains.

• Publication year

  2025

• Venue

  International Conference on Information and Knowledge Management

• Publication date

  2025-11-10

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/3746252.3760811
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An analysis of large language models: their impact and potential applications

  2024cited by this paper
• The FAIR Assessment Conundrum: Reflections on Tools and Metrics

  2024cited by this paper
• LLM for Data Management

  2024cited by this paper
• FAIR sharing of health data: a systematic review of applicable solutions

  2023cited by this paper
• A Survey on Evaluation of Large Language Models

  2023cited by this paper
• FAIR degree assessment in agriculture datasets using the F-UJI tool

  2023cited by this paper
• GPT-4 Technical Report

  2023cited by this paper
• Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4

  2023cited by this paper
• FAIR assessment tools: evaluating use and performance.

  2022cited by this paper
• A Comprehensive Comparison of Automated FAIRness Evaluation Tools

  2022cited by this paper
• FAIR for AI: An interdisciplinary and international community building perspective

  2022cited by this paper
• From Conceptualization to Implementation: FAIR Assessment of Research Data Objects

  2021cited by this paper
• Be FAIR to your data

  2020cited by this paper
• Language Models are Few-Shot Learners

  2020cited by this paper
• The Australian Research Data Commons

  2019cited by this paper
• FAIRshake: toolkit to evaluate the findability, accessibility, interoperability, and reusability of research digital resources

  2019cited by this paper
• Evaluating FAIR maturity through a scalable, automated, community-governed framework

  2019cited by this paper
• The FAIR Guiding Principles for scientific data management and stewardship

  2016cited by this paper
• The Fourth Paradigm: Data-Intensive Scientific Discovery [Point of View]

  2011cited by this paper

• No citing papers are available for this paper.