Metrics for quantitative evaluation of FAIR Data

The FAIR Guiding Principles for scientific data management and stewardship (https://www.nature.com/articles/sdata201618) say that all scholarly outputs should be Findable, Accessible, Interoperable, and Reusable by both people, but more importantly, by computers. To achieve this goal, the FAIR Principles state that:

To be Findable:

F1. (meta)data are assigned a globally unique and persistent identifier F2. data are described with rich metadata (defined by R1 below) F3. metadata clearly and explicitly include the identifier of the data it describes F4. (meta)data are registered or indexed in a searchable resource

To be Accessible:

A1. (meta)data are retrievable by their identifier using a standardized communications protocol
A1.1 the protocol is open, free, and universally implementable
A1.2 the protocol allows for an authentication and authorization procedure, where necessary A2. metadata are accessible, even when the data are no longer available

To be Interoperable:

I1. (meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation.
I2. (meta)data use vocabularies that follow FAIR principles I3. (meta)data include qualified references to other (meta)data

To be Reusable:

R1. meta(data) are richly described with a plurality of accurate and relevant attributes R1.1. (meta)data are released with a clear and accessible data usage license R1.2. (meta)data are associated with detailed provenance R1.3. (meta)data meet domain-relevant community standards

The principles apply to a wide range of entities, including data (of any sort, physical or digital objects), metadata (information about data), and infrastructure (e.g. analytical tools, pipelines, search services).

The FAIR Principles have seen rapid adoption worldwide by all stakeholders, including journals and funding agencies, and within a very short time, data publishers became under increasing pressure to adhere to the FAIR Principles. As a result, some publishers claimed to already “be FAIR”. While this was clearly a false claim, it was also impossible to refute, because there was no way to objectively measure “FAIRness”.

To resolve this situation, a group of stakeholders (journal editors, data publishers, bioinformatics researchers, technicians, and data
scientists) created the FAIR Metrics Authoring Group. This group gave itself the task of defining objective Metrics by which a digital object could be evaluated against the FAIR Principles. In addition, the group laid-out the rubric within which new Metrics could be defined by the wider stakeholder community.

This article (https://www.nature.com/articles/sdata2018118) describes the rubric, and the first set of 14 FAIR Metrics defined by this authorship group. It is intended to recruit community involvement in evaluating and refining the Metrics, to ensure that they have broad support by all stakeholders.