# Layer 4 · Data

URL: https://justuju.org/layer/data/

The text, code, images and feedback that models learn from, and how it's collected, cleaned and licensed.

Gives the layer above: Training corpora and feedback data that shape what a model can do.
Needs from the layer below: Storage and compute from Cloud & Infrastructure to collect, filter and deduplicate at scale.

## Concepts

- [Pretraining data](https://justuju.org/concept/pretraining-data/): The huge collection of text and other media a model first learns from, and how it is filtered.
- [Data curation and mixing](https://justuju.org/concept/data-curation/): Removing copies and junk from raw data, then choosing how much of each kind of data a model trains on.
- [Synthetic data](https://justuju.org/concept/synthetic-data/): Training data generated by models or programs rather than collected from people.
- [Human feedback and labelling](https://justuju.org/concept/human-feedback-data/): Example answers and side-by-side ratings written by people, used to teach a model to follow instructions and to prefer better answers.
- [Data rights and provenance](https://justuju.org/concept/data-provenance/): Tracking where training data came from, what licence it carries and whether its owners opted out, and the legal questions still open.

---
Justuju · https://justuju.org/ · Content CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote with a link to the page.