Structure of Scientific Revolutions

Category: Books

It’s easy to get carried away with a beautiful abstract theory, so I look to appreciate the conditionals on the structure of scientific research that will let me tune predictions to an instance. That said, many of these concepts carry cleanly to science that I’ve seen.

That said, applications abound. The easy examples from machine learning come out of questioning the basic methodology, as well as questioning the things that are worth spending time thinking about and studying.

My first major critique is the (relative) unimportance of understanding when it comes to the deep learning literature. There’s an objective function that looks like performance on a dataset, where a nicer objective function would have a very clear element that was about understanding the nature of the algorithm, whether it be the structure, the types of function that are fit, the properties of important alterations to the algorithm and reasons for their success or failure, etc.

It’s almost as if the space is insufficiently experimental. It would be wonderful to have a concrete working mental model for how these functions are created that we could use to intuitively make changes.

But this is also a local success. It’s likely that there are more fundamental changes to the model (akin from going from feed-forward nets to convnets) that should be taking precedent over improvements to the existing thing. And there’s the real tradeoff between marginal improvements on a current solution and fundamental changes that are much less likely to succeed but much more important when they do succeed.

This is also why it’s important to understand machine learning generally as well as networks. There are general principles that are much better evinced by other models, and the standard for understanding a change’s impact with a mental model of the system is in a much better place.

That said, Kuhn - he has this concept of paradigm, which I’ve mapped to (in my language) local equilibria. But he may say something importantly different, which I assume away. That said, the economic model (god, this really is how-to-think like) reaped rewards in explanation which may mean that it reaps rewards in understanding reality.

The major (and obvious, so extremely important) insight is that there are assumptions that one has to make before any research can be productive. Those assumptions on the methodology and choice of direction are often more impactful on the value of the research than whether or not the research is done well within the paradigm. When those assumptions remain hidden it becomes impossible to condition on the situations that the assumptions were built around. It also creates intellectual “dead space” where valuable research could be but is not occurring, which the space will pivot to once they see weaknesses or limits to their methods. But if they cannot see those limits, it’s likely that those spaces go undiscovered. Limits require an expectation of performance.

I should think about the interrelatedness of fields as well, and watch as history cuts them up into “clean” classes and so defines away large swaths of research space.

Source: Original Google Doc