Empiricism & LLMs

For those of you who have not made use of large language models (LLMs) to accelerate your own personal learning, I offer these ideas as a positive inspiration. In my experience, provided that I stay away from topics enmeshed within present-day curated narratives, closed LLMs such as ChatGPT can assist with effective learning. Hard sciences such as chemistry or physics and mathematical questions about calculus or statistics are particularly easy to query. In contrast, topics associated with ethical, psychological, environmental, sociological, political, or philosophical matters are more difficult to ask AI about without wading through supposed-tos and shoulds interlaced with almost every answer.

One thing that I've noticed in my own use of closed LLMs like OpenAI and Claude is that the more concrete my question--a particular math problem, one specific scientist's publications, etc.--the more reliable the responses I get back tend to be. Conversely, the more abstract my query, the more I ask generative AI to generalize across societies and times, the less it tends toward reliable outputs.

Here is an example of one way to guide an LLM to navigate an abstract concept. Recognize that the underlying claim, that scientific empiricism is inductive (moves from particulars to a general abstraction) and hypothesis/theory generation is deductive (infers specifics from general principles) guides the response.

LLM used: ChatGPT 4o (OpenAI)

Prompt that I ran: "Develop the idea that the process of scientific empiricism is inductive, and that hypothesis and theory generation is deductive."

Run Date: 10/24/2024

Everything below the following line is ChatGPT's response to the above prompt.

The process of scientific empiricism, rooted in observation and experimentation, is largely inductive, while hypothesis and theory generation in science are deductive processes. Both forms of reasoning are crucial to scientific inquiry, but they serve different functions within the scientific method.

Inductive Reasoning in Scientific Empiricism

Inductive reasoning is the process of drawing general conclusions from specific observations. In the context of scientific empiricism, this means that scientists observe particular instances or phenomena and then use those observations to formulate broader generalizations or patterns. For example, if a scientist notices that a certain chemical reaction consistently occurs under specific conditions in multiple trials, they may infer that this reaction is a general rule.

This inductive process forms the basis of scientific empiricism:

1. Observation: Scientists gather data through experiments or observations.
2. Pattern Recognition: They identify recurring patterns in the data.
3. Formulation of General Principles: From these patterns, they propose general principles or laws that describe the observed phenomena.

The strength of inductive reasoning lies in its capacity to discover patterns in nature, but its conclusions are inherently tentative, as new observations can potentially refute previous generalizations. Thus, inductive conclusions in science are often treated as probabilistic rather than certain.

Deductive Reasoning in Hypothesis and Theory Generation

Once general principles are established through inductive reasoning, deductive reasoning comes into play when generating hypotheses and developing theories. Deduction moves from the general to the specific, starting with an established theory or principle and applying it to make specific predictions.

In the scientific method:

1. Theory: A well-established, general framework that explains a broad set of observations (e.g., the theory of evolution).
2. Hypothesis: A specific, testable statement that derives from the theory (e.g., “If species evolve through natural selection, then we should find transitional fossils in the geological record”).
3. Prediction and Testing: Deductive reasoning helps scientists formulate hypotheses by taking the general principles of a theory and applying them to predict outcomes in specific cases.

The beauty of deduction is that if the premises (the theory and hypotheses) are true, then the conclusion must also be true, allowing scientists to test hypotheses in controlled experiments. Hypothesis testing, if successful, reinforces the theory from which it was derived; if the hypothesis is falsified, it may lead to a revision or rejection of the theory.

Integration of Induction and Deduction in Science

Induction and deduction are complementary processes in scientific inquiry. Scientists use induction to discover new patterns or principles based on empirical evidence, which then leads to the generation of theories. From these theories, scientists use deduction to create testable hypotheses, guiding further experimentation.

In this cycle, science balances inductive discoveries with deductive testing, allowing for the growth of knowledge through the refinement and validation of theories. While empirical observation (induction) lays the foundation, hypothesis and theory generation (deduction) structure and guide scientific investigation in a methodical way.

Ok, thank you ChatGPT.

The underlying idea here, around which the above was generated, is that the scientific process seeks to abstract what is generally true out of the particulars we observe. In data science, this is akin to finding a "signal in the noise." When we inductively abstract patterns from data, we engage in a bottom-up approach -- we find something consistent in the relationship of local data that we expect to be consistent with similar data gathered other places and times. Our models explain some portion of observed variance, and these understandings naturally lead, when unhindered and uncorrupted, to hypotheses about what else we might find had we sufficient data to analyze. 

In the public health domain, we attempt to generalize particular findings to broader populations, and with this understanding, so can you.   

The only other thing I will add to this is the following short animation. It's how I sometimes feel about myself after reading what generative AI completed in ten seconds. Then again, its output was formulaic and, in a literal sense, derivative. Keep in mind that LLMs do not understand anything -- but an LLM can save time, much like having access to an unoriginal research assistant. Used appropriately, generative AI can provide base content to learn by and augment human capabilities.