It is not better in natural sciences. Alzheimer’s disease treatment based on the amyloid plaque hypothesis being the prime example. Basically researchers closed ranks and doled out grant money disproportionately to the amyloid hypothesis researchers.
This is sort of like adding momentum to a gradient descent algorithm, though. It is rational congeal support around a plausible hypothesis to see if it pans out rather than pursue all hypotheses (whatever that would even mean) in a desultory fashion. In the case of amyloid plaques there may have been some academic misconduct involved as well.
Sometimes I feel like non-scientists have this idea that scientists should be super-rational actors and never be bamboozled or be wrong, but frankly, that isn't realistic. Scientists are fallible, have finite time, and are subject to social trends and pressures like the rest of us. Expecting that they always get it right is unrealistic and not good for science.
Scientists should not shut out other ideas for two decades even though there were serious challenges to the hypothesis the whole time. Part of being a good scientist is admitting your chosen hypothesis isn’t correct and that it’s time to create a new theory based on the experimental evidence available. Otherwise, it’s just a research based cult.
This almost never happens, to my knowledge. Most fields have people pursuing alternative hypotheses even when one thing is in vogue. For example, in the case of Alzheimer's Disease there were and are a lot of other threads of research and I would be that your typical AD scientist would have had a good grasp of many of them regardless of the research they were focusing on.
Yes. At least there is a plausible argument for it in many contexts. Imagine a suite of ten hypothesis, one of which is correct but each of which requires 6 effort units to prove or disprove or discover or whatever. If we only have a 10 units of effort to devote to research, we won't get anywhere if we divide them equally but we can eliminate each hypothesis if we work serially.
In reality I would expect the effect to be even more pronounced because progress on a hypothesis is almost certainly non-linearly related to the number of people working on it (with some point of diminishing return, of course). It is totally reasonable for a community to more or less work on proving or eliminating the few most reasonable hypotheses at a time rather than to spread themselves over the (frankly enormous) space of hypotheses relevant to a particular area.
I'd hardly argue that our system of allocating scientific research effort is perfect, but how could it be?
> Imagine a suite of ten hypothesis, one of which is correct but each of which requires 6 effort units to prove or disprove or discover or whatever. If we only have a 10 units of effort to devote to research, we won't get anywhere if we divide them equally but we can eliminate each hypothesis if we work serially.
A model of research without evidence demonstrating its accuracy is not convincing.
You might claim that we could never have discovered the Higgs without such coordinated efforts, and I could counter that without such coordination, we might have sooner discovered and advanced alternative paths that require considerably less funding and coordination, like wakefield accelerators.
I don't think science that advances by stepwise consensus will outperform random, independent search in general, but only in very limited scenarios.
> It is totally reasonable for a community to more or less work on proving or eliminating the few most reasonable hypotheses
Only if "the community" consists of largely independent thinkers that reach their own conclusions, and are not influenced by fads or celebrity personalities. That's the only way to actually ascertain "the most reasonable hypotheses", and unfortunately, scientists are not immune to such influences.
I think current funding mechanisms for science select for non-independent thinkers. At the beginning of the scientific era, most researchers were funded by patrons, which supported much more independent research (at least, independent of other researcher's opinions, not the patron).
