Hypothesis-Driven Hunting

Author: Roger C.B. Johnsen

Introduction

Hypothesis-driven hunting is a threat hunting approach where the hunter starts with a testable idea about possible adversary behaviour and then uses data to validate, weaken or reject it.

The hypothesis gives the hunt direction. It tells the hunter what behaviour to look for, which data sources may be relevant, what scope should be used and what kind of evidence would matter.

A weak hunt often starts with a vague question:

Do we have attackers in the environment?

That is too broad to be useful. A stronger hypothesis is more specific:

An adversary may be using valid accounts to move laterally from workstations to servers outside normal administrative patterns.

This gives the hunter something to test. The value of hypothesis-driven hunting is not that the hunter already knows the answer. The value is that the hunter has a structured question that can be investigated. A good hypothesis should create direction without pretending to be certainty.

A hypothesis is not a conclusion. It is a disciplined way to decide what to test next.

– Roger Johnsen

What Hypothesis-Driven Hunting Is

Hypothesis-driven hunting is built around a simple idea: the hunter makes a testable statement about possible adversary behaviour, then investigates whether the environment supports or weakens that statement.

The hypothesis may come from threat intelligence, previous incidents, red team findings, ATT&CK techniques, anomaly-driven hunting, detection gaps, business risk, crown jewel analysis or analyst experience.

The important part is that the hypothesis must be testable.

A hypothesis-driven hunt needs a few things to work well. The hypothesis must be testable. The scope must define which systems, users, time ranges and data sources are included. The hunter also needs to know what kind of evidence would support, weaken or reject the hypothesis, and what context is required to decide whether the evidence matters.

The outcome does not have to be binary. A hunt may support the hypothesis, weaken it, reject it, remain inconclusive or lead to a better question.

The hypothesis does not have to be perfect at the start. It often improves during the hunt. That is normal. Hunting is investigative work. The hunter starts with a structured assumption, tests it against data and adjusts as evidence appears.

Why Hypotheses Matter

Without a hypothesis, a hunt can easily become browsing. The hunter opens logs, searches for interesting events, follows a few oddities and eventually stops when time runs out. That may still produce findings, but it is difficult to repeat, measure or explain.

A hypothesis helps prevent that. It gives the hunt a purpose. It also helps the hunter decide what not to do. This matters because security telemetry is enormous. Without scope, a hunter can drown in data.

A useful hypothesis helps answer:

• What behaviour are we looking for?
• Why do we think this behaviour may matter?
• Which systems, users or assets are in scope?
• Which data sources can support the hunt?
• What would support the hypothesis?
• What would weaken the hypothesis?
• What would make the result inconclusive?
• What should happen if the hypothesis is supported?

This is why hypothesis-driven hunting is one of the most practical ways to structure threat hunting.

It turns curiosity into an investigation.

What Makes a Good Hypothesis

A good hunting hypothesis should be specific enough to guide the hunt, but not so narrow that it only searches for one known indicator.

It should usually include:

A strong hypothesis might look like this:

An adversary may be using valid accounts for lateral movement by authenticating from user workstations to servers outside normal administrative patterns, followed by remote service creation or suspicious process execution.

This hypothesis is useful because it suggests:

• identity data
• endpoint telemetry
• server authentication logs
• remote service creation events
• administrative activity baselines
• peer group comparison
• follow-up pivots

A weak hypothesis would be:

Attackers may be using valid accounts.

That may be true, but it is too broad. It does not give enough direction. A hypothesis should narrow the investigation without closing the hunter’s mind.

Sources of Hypotheses

Hypotheses can come from many places.

This is where hypothesis-driven hunting connects naturally to TaHiTI. Threat intelligence can create the hunting idea. The hypothesis turns that idea into something testable.

From Idea to Hypothesis

Many hunting ideas start too vague. For example:

We should hunt for ransomware.

That is not a useful hypothesis. Ransomware is an outcome, not a specific behaviour to test. A better path is to translate the idea into behaviour:

Ransomware operators often prepare by accessing backup systems, disabling recovery options or moving laterally using privileged accounts.

Then the hunter can create a testable hypothesis:

An adversary may be preparing for ransomware activity by using privileged or service accounts to access backup infrastructure outside normal administrative patterns.

Now the hunt has direction. The hunter can define:

• relevant accounts
• backup systems
• authentication paths
• administrative tools
• process execution
• change windows
• expected behaviour
• suspicious deviations
• escalation criteria

That is the difference between a topic and a hypothesis:

• A topic says what the hunt is about.
• A hypothesis says what the hunt will test.

Hypothesis-Driven Hunting Process

A hypothesis-driven hunt can be structured as a simple process.

This process should be disciplined, but not rigid. The hunter should not force the data to fit the hypothesis. The hypothesis is a tool for investigation, not something that must be proven.

Evidence: Supporting, Weakening and Inconclusive

A common mistake is to think that a hypothesis-driven hunt has only two outcomes:

Confirmed or not confirmed.

Reality is usually more nuanced.

A rejected hypothesis can still be valuable. It may show that a technique is not currently visible in the environment, that controls are working, or that the original risk assumption was weak. An inconclusive result can also be valuable. It may reveal that logs are missing, retention is too short, endpoint coverage is incomplete or the available data does not support the hunt. That is not wasted effort. That is knowledge.

A hypothesis-driven hunt does not fail because the hypothesis is rejected. It fails when the team learns nothing from the test.

– Roger Johnsen

Practical Example: PowerShell Execution

Consider a hunt based on suspicious PowerShell usage. A weak hypothesis might be:

Attackers may be using PowerShell.

That is too broad. PowerShell is used for many legitimate administrative tasks. A better hypothesis might be:

An adversary may be using PowerShell to execute encoded commands from user workstations where PowerShell is not normally used for administration.

This is more useful because it defines behaviour, scope and expected deviation.

Prepare

The hunter defines:

• user workstations in scope
• expected administrative hosts
• PowerShell logging availability
• command line logging quality
• script block logging availability
• normal administrative patterns
• time range
• known management tools
• existing detections

The hunter also defines what evidence would matter:

• encoded commands
• suspicious parent processes
• unusual user context
• network connections after execution
• downloaded scripts
• execution from office applications or browsers
• activity outside normal administration patterns

Execute

The hunter queries endpoint telemetry and PowerShell logs.

Possible pivots include:

• parent process
• command line
• user
• host
• script content
• network destination
• file creation
• related authentication
• timeline around execution

The hunter compares findings against normal administrative behaviour.

Act

Possible outputs include:

• confirmed suspicious PowerShell activity
• rejected hypothesis with documented reasoning
• improved baseline of legitimate PowerShell use
• detection logic for encoded PowerShell from unusual parent processes
• recommendation to enable missing PowerShell logging
• triage guidance for analysts
• follow-up hunt for related credential access or lateral movement

The value is not only whether malicious PowerShell is found. The value is that the organisation now understands the behaviour better.

Hypothesis-Driven Hunting and Detection Engineering

Hypothesis-driven hunts can produce valuable detection engineering output. A hunt may identify behaviour that should become a detection. It may also reveal that an existing detection is too narrow, too noisy or missing important context.

Possible outputs include:

• new detections
• improved detections
• triage guidance
• enrichment requirements
• suppression logic
• visibility gaps
• validation tests
• baseline updates
• follow-up hunts

The team should ask:

A hunt query is not automatically a production detection. The query may be exploratory, expensive, broad or dependent on manual context. Turning hunt output into detection engineering requires tuning, validation, testing and ownership.

Where Hypothesis-Driven Hunting Fits

Hypothesis-driven hunting fits naturally with the other frameworks in this book.

A hypothesis gives the hunt something to test, but other frameworks help shape where the hypothesis comes from, how it is executed and what happens afterwards.

For example, TaHiTI may provide the intelligence-driven trigger, ATT&CK may help describe the relevant behaviour, and the Pyramid of Pain may help the hunter ask whether the hypothesis is focused on something durable or something the adversary can easily replace.

PEAK then gives the hunt a lifecycle:

Prepare → Execute → Act with Knowledge

In that lifecycle, the hypothesis is not the whole hunt. It is the steering mechanism. It helps the hunter decide what to test, what evidence matters and how the result should be interpreted.

What Usually Goes Wrong

A common failure pattern is simple: the team starts with a threat topic, calls it a hypothesis and begins querying.

Hunt for lateral movement.

That is not a hypothesis. It is a topic.

A useful hypothesis explains what kind of lateral movement may be happening, where it may appear, how it may differ from normal behaviour and what evidence would matter.

Several problems are common:

A hypothesis-driven hunt should create focus, not tunnel vision.

Working Position for This Book

For this book, hypothesis-driven hunting is best treated as a practical approach for turning a threat idea into a testable investigation.

It helps answer a simple question:

What do we believe may be happening, and how can we test it?

The value of hypothesis-driven hunting is focus.

It helps the hunter avoid aimless searching, define relevant data, test assumptions and produce a result that can be explained.

But the hunter must remain honest.

The goal is not to prove the hypothesis. The goal is to learn whether the evidence supports it, weakens it, rejects it or points to a better question.

Resources

• MITRE ATT&CK
• A Guide to Threat Hunting: Hypothesis-Driven Approach
• Pyramid of Pain by David Bianco
• ThreatHunting.org
• DEF-TaHiTI Threat Hunting Methodology

Revision