How does data become intelligence?
It all starts
with the data.
Superintelligence will raise the most consequential scientific questions of our time, transforming the potential for progress and shattering the limits of human capability. Much of that progress will likely come from increasingly capable models. But a model is a reflection of the data it is trained on. And while a lot of research examines architectures, training methods, and model optimizers, everything that a model could theoretically learn is governed by its data and experiences.
If we can understand how data gives rise to model capabilities, we can dramatically expand our ability to shape, control, and customize intelligence. In the last three years, data curation methods such as filtering, deduplication, mixing, and synthetic data have become major levers of progress. But the field's understanding of why they work has lagged far behind. Practitioners know which recipes help, but less often why, or whether they will still help at the next scale, in the next domain, or for the next model. We're still challenged to predict what a dataset will teach a model without spending the compute to train one, so data decisions are largely settled by trial and error, one training run at a time.
Data research is orphaned between the people resourced to do it and the people motivated to share it. The organizations with the data, compute, and infrastructure to study data at scale have no incentive to disseminate their research. While academic researchers who have the incentive to publish, typically lack the resources to run data experiments at breadth and scale. And across the field, data work has traditionally been considered plumbing rather than science.
Data research is the soul of DatologyAI. Pursuing our mission has allowed us to consistently innovate and develop greater understanding of the science of data, while also teaching us how much still remains unexplored. Today, we're expanding that commitment with Datology Labs, our open research initiative dedicated to defining the science of data. We believe ambitious science is best pursued collaboratively and in the open. Datology Labs will bring together talented researchers from Datology, academia, and beyond who share an obsession with data, and give them the freedom to pursue some of the most meaningful work of their lives.
Defining the science of data.
The more we learn about data, the more questions we discover. Some of these questions stem from the fundamental relationship between data and learning, while others challenge how we think about safety, alignment, and entirely new scientific domains. Discoveries that demand big swings also demand exceptional ambition, and an appetite for failure. We are sharing a non-exhaustive list of data problems that we stay excited about.
Studying the data life cycle.
Data has a life cycle that extends beyond preparing a training dataset. It is sourced and processed, then ordered and combined across stages of training. As a model learns from that data, information is absorbed into its weights, where it interacts with what the model has learned before and with the data it encounters later. These interactions ultimately shape the capabilities and behaviors that emerge. Models also inherit information from other models through distillation and synthetic data. Their capabilities and behaviors shape what data they produce and what humans create from them. Models write data back into the world, which can become part of the next model's training data.
We want to understand these interactions across the entire life cycle. Why do some examples reinforce previous learning while others interfere with it or cause forgetting? How does data order affect learning and training stability? How does data seen in different training stages interact with each other? And how should we curate data for distillation and synthetic generation to preserve and build on what previous models have learned?
Understanding how data shapes learning.
At Datology Labs, one of our bets is that it is possible to predict what a model will learn before we spend the compute to train it, through the in-depth study of data. Somewhere in a model's weights, data transforms into a model’s knowledge, capability, and behavior, but we still don't fully understand how. Understanding where information is represented and how behaviors and capabilities arise from training examples could help us close the gap between what data contains and what models are able to learn from it, allowing us to train models that are significantly more data efficient than the current generation.
To get there, we need new ways to study data itself. What makes some data more useful than other data? Can small-scale experiments predict large-scale model behavior? We want to develop measurements and visualizations that help answer these questions, alongside attribution methods that connect capabilities to training examples and forensic methods that investigate data provenance through model weights.
Safety and alignment through data.
Building superintelligence that is safe, faithful, and honest remains one of the most consequential problems of our time. As one might imagine, data remains an integral pillar in developing behaviors that are robust in the model. Understanding how data shapes these behaviors will be a central goal of our research.
Some of the challenges in this space include understanding where and how safe and unsafe behaviors live inside models. What data curation properties can help make safety inseparable from a model's useful capabilities? Does the timing of a data intervention change its robustness? Can seemingly benign data contain natural poisons that compromise safety? Can we identify the examples responsible for an unsafe behavior and design interventions that prevent it?
Taking the science of data to new domains.
Machine learning research has followed the same pattern again and again. When entering a new domain, the field rushes to models, architectures, optimizers, and training techniques. This sequence of events has now taken shape across vision, language, time-series, and multimodal systems. Enormous effort often goes into improving how models learn before we develop an equally rigorous understanding of what they learn from.
The next phase of AI systems is hitting consequential fields such as physics, materials science, and biology. While many of these domains are presently constrained by limited data, there remains an enormous amount of leverage in bringing the joys of data curation to new fields, from classic algorithms to new synthetic data approaches. Which principles of data curation carry over to these fields, and which need to thought again from scratch?
Work with us.
We are excited about the unknown and the uncertain. We deeply want to follow the unanswered questions and unexpected discoveries that come from data. Come have fun with us!
Spend time researching a frontier data research problem as a Datology Labs fellow. All we ask is that you are obsessed with data and willing to take risks.
Want to Partner?
We are also looking to partner with labs, companies, and universities interested in defining the science of data in their areas of expertise. Join us in defining the science of data. Together.
Partner with us
