There are several interesting items in this chapter from a CIA handbook on expertise and aggregating expert opinions, but one point raises an issue for those in the business of teaching “Critical Thinking”.
Experts are individuals with specialized knowledge…but that expertise does not necessarily transfer to other domains. A master chess player cannot apply chess expertise in a game of poker; although both chess and poker are games, a chess master who has never played poker is a novice poker player. Similarly, a biochemist is not qualified to perform neurosurgery, even though both biochemists and neurosurgeons study human physiology. In other words, the more complex a task, the more specialized and exclusive is the knowledge required to perform that task.
The point raised in this quote is the finding that people tend to have difficulty transferring domain specific expertise. Although it isn’t mentioned here, those difficulties also include the ability to recognize the very same problem that a person may find easy to solve in a domain familiar to him when the problem appears in a less familiar domain.
The question for courses in Critical Thinking is this: Are the skills taught in a typical CT course a form of expertise, and therefore involve limitations that other forms of expertise have in terms of the time it takes to acquire those skills and the limited scope of their applicability, or are the skills taught in CT courses lasting, of a general nature and therefore a (partial) solution the problem of crossing domains of expertise? Doubtless these are not exclusive cases, but the distinction is useful to focus attention on the philosophical issue of what the relationship is between logic and reasoning, and also to focus attention on the practical (and measurable) question of whether CT courses are effective. CT course descriptions often promise a lot, and it is a fair question to ask whether they deliver.
I have my doubts on both counts, but welcome contrary arguments and data.