...
[Hannah: I would say expertise depends a bit on which tool you use; most checklists and questionnaires are pretty low effort and self-explenatoryexplanatory. But some of the more automated tools require some (programming) skills]
...
[Hannah: Fieke pointed out something important; there are basically two kinds of tools for the FAIR assessment. One group assesses (often in a semi-automated way) the FAIRness of (meta)data which already has a persistent identifyer identifier (such as a DOI). The other group assesses FAIRness (often in the form of a survey, questionnaire or checklist) of (meta)data without persistent identifyeridentifier.]
Online self-assessment surveys
...
[Hannah: the 2022 paper does not recommend using offline tools, and I kind of agree. So maybe we don’t include this category at all? Especially because one of the links is dead anywaysanyway..]
FAIR assessment tools vary greatly in their outcomes. The FAIR Data Maturity Model (created by the Research Data Alliance, or RDA) aims to harmonise outcomes of FAIR assessment tools to make these comparable.
FAIR maturity evaluation system
FAIR Implementation Profiles (FIPs)
Based on the FAIR principles and sub-principles, they have created a list of universal 'maturity indicators'. Their work resulted in a checklist (with extensive description of al maturity indicators), which can be used to assess the FAIRness of your (meta)data.
FAIR maturity evaluation system
FAIR Implementation Profiles (FIPs)
Potentially: compare the community FIP with your own fingerprint. This gives an indication on whether you meet R1.3?
...
Hannah mentions the Data Maturity Model. This is also here on FAIRplus. There is also this Github from FAIRplus and the sheet for the actual assessment is here. Could be worrying: last update was last year.
Related: in the FAIRtoolkit they describe Data Capability Maturity Model:
...
[Hannah: I cannot really find a clear description on how to use it, only a huge excel file (for which you have to dig quite deeply into the GitHub, maybe we can link to it here if we include it https://github.com/FAIRplus/Data-Maturity/tree/master/docs/assessment ?]
Related: in the FAIRtoolkit they describe Data Capability Maturity Model:
Most recently, CMM has been adapted by the FAIRplus IMI consortium [7] to improve an organisation’s life science data management process, which is the basis for the method described here.
The FAIR data CMM method identifies 1) important organisational aspects of FAIR data transformation and management, 2) a sequence of levels that form a desired path from an initial state to maturity and 3) a set of maturity indicators for measuring the maturation levels.
e.g. Findability Maturity Indicators. Also describes some team requirements.
[Hannah; I think this is also more about assessing FAIR in an organization?]
Furthermore: FAIR Evaluator (FAIRopoly and FAIR Guidance) – text copied below.
FAIRopoly
FAIRopoly
As a task under the objectives of the EJP RD, we created a set of software packages – The FAIR Evaluator – that coded each Metric into an automatable software-based test, and created an engine that could automatically apply these tests to the metadata of any dataset, generating an objective, quantitative score for the ‘FAIRness’ of that resource, together with advice on what caused any failures (https://www.nature.com/articles/s41597-019-0184-5). With this information, a data owner would be able to create a strategy to improve their FAIRness by focusing on “priority failures”. The public version of The FAIR Evaluator (https://w3id.org/AmIFAIR) has been used to assess >5500 datasets. Within the domain of rare disease registries, a recent publication about the VASCA registry shows how the Evaluator was used to track their progress towards FAIRness (https://www.medrxiv.org/content/10.1101/2021.03.04.21250752v1.full.pdf). To date, no resource – public or private – has ever passed all 22 tests, showing that FAIR assessment is able to provide guidance to even highly-FAIR resources.
The FAIR evaluation results can serve as a pointer to where your FAIRness can be improved.
[Hannah; this tool is included in the section above already: semi automated tools]
FAIR Guidance [https://www.ejprarediseases.org/fair_guidance/]
...