Define Key Takeaways

Overview

The success criteria for your dataset, the decisions readers need to make, and the information they need, can be used to craft a compelling story that you can reinforce about your dataset across the Data Card. This story provides guidelines to frame answers and decide how much information to provide.

When people read Data Cards, they want to make very specific decisions, such as “Is this dataset suitable for my use case?”, “Can I allow others to use the dataset?”, or “How can I safely use this dataset without adding risk to my models?”.

Readers are incredibly adept at making dataset-related decisions within their contexts if they can access the right information efficiently. The importance or usefulness of information depends on the type of decision the reader needs to make, and their background. For instance, when deciding whether or not to use a dataset, a compliance officer might look at the licenses associated with it, but an engineer will look at the technical stack. Both readers ask the same question, but the answers they expect are very different.

These decisions will help you decide what you want readers of your Data Card to walk away with, and the kind of accurate, robust, and well-organized information that needs to be documented in the Data Card. So Data Cards must comprehensively describe your data sets so readers can make decisions confidently. However, it’s impossible to determine all the possible decisions that readers of your Data Card will need to make. Start with a focus on the most common and impactful decisions around your dataset that readers are likely to make, and branch out from there.

Key Takeaways

• It’s important to frame your transparency artifact around your readers because different readers have different information needs.
• It is equally important to define your dataset’s success so that your Data Card can support and augment it.

Actions

Use this to identify the most critical decisions that readers will need to make. These decisions will help you decide what you want readers of your Data Card to walk away with, and the kind of accurate, robust, and well-organized information that needs to be documented in the Data Card.

1. Define the success criteria. Before you start your Data Card, clearly define the success criteria for your dataset. Is your dataset successful when:
   • The usage of the dataset increases? If so, by whom, and in what contexts?
   • A specific team or client uses the dataset for a particularly challenging use case?
   • The dataset is used for a specific purpose, for example, as a benchmark for advancing accessibility research?
   • Your dataset clears a tough review process so you or others can start to use it?
2. Identify critical reader decisions. Use the success criteria to articulate the most important decisions that readers will need to make. Your data card should help readers create a working mental model of the dataset, which, in turn, will allow them to make these decisions.
3. Translate decisions into content. Finally, consider the different information types, such as objective (e.g. feature count) and subjective (e.g. intended use), and the level of detail which will help readers make the decision.

Considerations

• Do the key takeaways of your Data Card line up with decisions that readers need to make?
• Do key takeaways align with fidelities suitable to the reader’s domain expertise or data fluency?