Automating Authenticated Crawling and IDOR Discovery with Katana + Burp

· falasi.net


IDOR Automation

One part of web testing I’ve always found repetitive is crawling authenticated applications and then checking the resulting requests for authorization issues.

You can manually click through an application with Burp open, but on a large dynamic site that gets old pretty quickly.

Recently I started combining Katana with Burp Suite and Auth Analyzer.

The idea is simple:

1Authenticated Session
23      Katana
45   Burp Proxy
67  Auth Analyzer
89Potential authorization issues

Katana handles the crawling.

Burp captures the traffic.

Auth Analyzer helps identify requests where changing or removing authentication may produce an interesting result.

Then I manually investigate what it finds.

Why Katana? #

Traditional crawlers work well when an application is mostly links and server-rendered pages.

Modern applications are different.

A lot of navigation happens through JavaScript, API requests, dynamically generated routes, buttons, and XHR/fetch requests.

Katana can run using a headless browser, which makes it much more useful against applications like this.

For example:

1katana \
2  -u https://target.example \
3  -headless \
4  -proxy http://127.0.0.1:8080 \
5  -d 5 \
6  -jc \
7  -xhr

Burp is listening on 127.0.0.1:8080, so the requests discovered during the crawl pass through Burp just like normal browser traffic.

For authenticated applications I can provide the session cookies or use an authenticated Chromium session.

That means Katana isn't just crawling the public application.

It's crawling what my authenticated user can actually reach.

Let Burp See Everything #

This is the part I like.

Instead of having Katana crawl into its own output file and reviewing everything separately, I proxy the traffic through Burp.

Now the crawl automatically populates my normal Burp workflow.

 1Katana
 2 3   │ authenticated requests
 4 5Burp Proxy
 6 7   ├── HTTP History
 8   ├── Logger
 9   ├── Repeater
10   └── Auth Analyzer

I can still inspect requests manually, send interesting ones to Repeater, search through history, or use any other Burp tooling I normally use.

Katana basically becomes another source feeding requests into Burp.

Adding Auth Analyzer #

This is where the workflow becomes more interesting.

I'm currently using Auth Analyzer instead of Autorize.

Both are trying to solve a similar problem, but right now I prefer Auth Analyzer because the UI is simple and I found it very quick to configure.

I give it the authentication states I want to compare and then let it analyze the requests moving through Burp.

So while Katana crawls the authenticated application, Auth Analyzer is seeing those requests at the same time.

Instead of:

 1crawl application
 2 3export URLs
 4 5find APIs
 6 7manually replay requests
 8 9change authentication
1011compare responses

I get something closer to:

1Katana discovers endpoint
23Request goes through Burp
45Auth Analyzer tests authorization context
67Interesting difference gets flagged
89Manual verification

That's a much better workflow for me.

Looking for IDORs #

Obviously this doesn't magically find IDORs.

The important part is still understanding the application and verifying the behavior manually.

But it does help answer a useful question:

Which authenticated requests are worth looking at more closely?

Imagine Katana discovers requests like:

1GET /api/users/18492/profile
2GET /api/accounts/8301/settings
3GET /api/projects/771/members
4GET /api/documents/92281

Those are immediately more interesting to me than another request for:

1GET /assets/logo.svg

If Auth Analyzer notices that a request behaves unexpectedly when the authentication context changes, I now have something worth sending to Repeater and investigating.

From there I can test things like:

1User A → Object A
2User B → Object A
3Unauthenticated → Object A

and determine whether there is actually an authorization problem.

Automation narrows the search.

Manual testing proves the vulnerability.

Crawling Is Recon Data #

Another reason I like this setup is that the crawl itself becomes useful data.

I'm already building my recon workflow around keeping historical data instead of treating every scan as disposable output.

Authenticated crawling can fit into the same model.

Store things like:

1endpoint
2HTTP method
3parameters
4response status
5content type
6authentication state
7first seen
8last seen
9source

Now you can start comparing crawls over time.

Maybe an endpoint appears that didn't exist last week.

Maybe an authenticated API suddenly becomes accessible without the expected authorization.

Maybe a new numeric object identifier starts appearing everywhere.

Maybe Katana discovers an API route that none of your other recon tools knew existed.

That's where crawling becomes more than collecting URLs.

Don't Let the Crawler Destroy Things #

One thing I don't want is a crawler blindly clicking everything.

Authenticated applications often contain actions like:

1Delete Account
2Remove User
3Rotate API Key
4Logout
5Cancel Subscription
6Delete Project

So scope matters.

I keep crawling restricted to the target application and avoid destructive routes whenever possible.

Automation should increase coverage without creating unnecessary risk.

Especially when testing production systems.

The Workflow #

Right now my workflow looks roughly like this:

 1                ┌──────────────┐
 2                │ Authenticated │
 3                │    Session    │
 4                └──────┬───────┘
 5 6 7                ┌──────────────┐
 8                │    Katana    │
 9                │ Headless/JS  │
10                └──────┬───────┘
111213                ┌──────────────┐
14                │  Burp Proxy  │
15                └──────┬───────┘
1617             ┌─────────┴─────────┐
18             │                   │
19             ▼                   ▼
20      ┌─────────────┐      ┌──────────────┐
21      │ HTTP History│      │ Auth Analyzer│
22      └─────────────┘      └──────┬───────┘
232425                         Interesting Requests
262728                           Manual Testing
293031                             IDOR / BOLA?

Nothing here is particularly complicated.

That's actually why I like it.

Katana does what it's good at: crawling.

Burp does what it's good at: capturing and manipulating HTTP traffic.

Auth Analyzer handles the repetitive authorization comparisons.

I spend my time looking at the requests that deserve human attention.

A crawler doesn't find the IDOR for you.

It gives you more opportunities to find one.

last updated: