Exploratory data analysis (EDA) is an approach used in statistics and data science, often leading to summarization and visualization of a dataset. More colloquially, EDA is to take a dataset and "roll around in it" in order to find interesting questions. This approach is often contrasted with hypothesis testing: having a question, a possible answer, and then explicitly testing for its validity. Which is better?
The famed American statistician John Tukey wrote the canonical book on Exploratory Data Analysis of the same name. In it, Tukey argued that too much emphasis was placed on statistical hypothesis testing, and instead more emphasis needed to be placed on EDA and using data to suggest the next hypothesis to test. Further, he thought that confusing the two types of analyses and using them on the same set of data can lead to bias. He often posed exploratory data analysis against "confirmatory data analysis".
Like any tool, the one you choose depends on the need. Even if the need is to make dinner you wouldn't sous vide a whole hog, just like you wouldn't barbecue a delicate fish. So even Tukey would agree that the statistical approach you use depends on the task at hand.
But the bigger question about exploratory data analysis is whether you need to know the question before you ask it. Can the question find you? Exploration is the act of traveling through an unfamiliar area in order to learn about it. Whether traveling through a dataset or a new country, exploration leads to discovery. And discovery, my friends, is the whole point.
Frequent readers of this site might have a hard time summarizing the most common article topics shared here: science, technology, software, web3, random musings? I consider Heureka Labs a bit like exploratory data analysis — traveling through new ideas, not exactly sure where I'm going or what the question is, until I discover exactly what I'm looking for.
Therein lies the beauty and the power of EDA. Creativity is a combinatorial art of making connections between different ideas. As such, it requires exploration and curiosity. Mixing and re-mixing, finding new combinations, until something new and wonderous emerges.
You don't always need to know where you're going to find your way to the place you need to be.