Unauthenticated, authenticated, or source-assisted?

Fast CREST penetration testing › Authenticated testing

If your application has a login, the single most consequential scoping decision you will make is whether the tester gets to use one. It is more consequential than the day count, the provider's accreditation or the tooling they run, and it is routinely decided by accident, by whoever filled in the scoping form.

The three depths

Unauthenticated, or black box

The tester works from outside with no credentials, as an anonymous internet user would. This covers the front door: the login mechanism itself, registration and password reset flows, anything exposed before authentication, the perimeter and its configuration, and information leakage from error pages, headers and forgotten endpoints.

It is genuinely useful and it is the right test for an external infrastructure range or a brochure site. For an application with logins it covers the smallest part of the attack surface, because nearly everything your application does happens after somebody has signed in.

Authenticated, or grey box

The tester is given working accounts, one for each user role, and tests what can be done from inside. This is where the findings that matter usually are: access control failures, data belonging to other customers, functions reachable by roles that should not have them, injection in authenticated parameters, flawed business logic in workflows that only exist once you are logged in.

For any application with users, this is the default that should be assumed unless there is a specific reason not to.

Source-assisted, or white box

The tester additionally gets source code, architecture documentation, or both. Tends to move effort from discovery into depth: fewer hours working out how the thing is built, more hours on the parts most likely to be wrong. Finds classes of issue the other two struggle with, particularly in cryptography, authorisation logic spread across many files, and code paths that are hard to reach from the interface but reachable by a determined attacker.

It does not usually reduce the day count. It improves what the same count produces.

Why role count is the multiplier

Here is the part that explains the price difference between two quotes for "a penetration test of our web application".

Testing one role means checking what that role can do and whether any of it is broken. Testing four roles means that, four times, and then the part that actually matters: the relationships between them.

Horizontal access control

Can customer A reach customer B's data? Same privilege level, different tenant or account. This is the most commonly found serious flaw in business applications and it is invisible to an unauthenticated test, because you need two accounts at the same level to test it at all.

Vertical access control

Can a standard user reach something only an administrator should? Often not through the interface, which hides the button, but through the request the button would have made. An interface that hides a function and an API that still honours it is one of the most reliable findings in the field.

Role transition and state

What happens when a user is downgraded, suspended or removed mid-session? Does an invitation token for a privileged role work for anyone who receives it? Does a session issued before a permission change still carry the old permissions?

None of these three can be tested with a single account, which is why "we gave them a login" is not the same as "the test was authenticated across our roles" and why a quote should say how many roles it covers.

What to give the tester, and when

Credentials that do not work on day one are the most common single cause of a test overrunning, and the cost is usually a full day. It is worth twenty minutes of somebody's time beforehand to prevent.

Access to prepare before the window opens

Item	Detail that matters
One account per role	Two at the same level for at least one role, so horizontal access control can be tested at all
Accounts that are actually provisioned	Logged into once by somebody on your side, with any first-run setup completed
Multi-factor arrangements	Either a shared enrolment route or MFA disabled for the test accounts, decided in advance rather than at 9am on day one
Representative data	Empty accounts hide most business logic. Populated ones find it
A route for locked accounts	Testing authentication locks accounts. Somebody needs to be able to unlock them the same hour
Secure transfer	Not plain email. Sending test credentials insecurely is exactly the class of thing the test exists to find

The objection, and the answer to it

The usual reason given for testing unauthenticated is realism: a real attacker would not be handed an account, so a test that is handed accounts is not realistic.

It is a reasonable instinct and it is wrong in nearly every case, for two reasons. The first is that a real attacker frequently does have an account, because on most applications anybody can sign up, or because credentials are phished, reused or bought. The second is that a penetration test is a time-boxed exercise and an attacker is not. Spending three of your five days getting the access an attacker would have obtained eventually anyway buys you a demonstration of something you already knew, at the cost of the days that would have found the access control flaw behind it.

If you want to know whether the perimeter holds, that is a defined and much cheaper piece of work. If you want to know whether your application is sound, give the tester accounts.

What the report should tell you either way

Whichever depth you choose, the report has to state clearly what was covered and what was not, so that nobody six months later mistakes a narrow test for a broad assurance. Specifically it should name which roles were tested and which were not, whether testing was authenticated, whether an API behind the interface was in scope separately, and what was deliberately excluded and why.

That last point is the one to insist on. A report which lists findings without stating its own limits reads as a clean bill of health to anybody who was not in the scoping call, and those are usually the people who make decisions about it later.