Are CTF games the best way to learn pentesting?

My short answer based on what I have seen…No.

Before you jump down my throat, let me say that CTF games are awesome challenges. The people behind root-me, hellboundhackers, hackthebox and the such, are doing an amazing job. Some challenges are just total brain fucks. But, This is not what pentesting looks like in real life.

My argument is based on a couple of points:

Different mindsets

Be it in pentests or in CTFs, you are not really competing against the app as much as you are against the human who designed it. The person who creates a CTF knows his app will be tested by stubborn individuals, and so will go out of his way to make this as hard as possible.
A developper on the other hand just wants to deliver his 10k lines of code on time. He does not think nor cares about hackers.
These two contradicting profiles make you approach the task diffrently, which brings me to my next point.

Different focus

A CTF designer will leave one tiny small door open to pwn his app: double encoding vulnerability, utf7 encoding bypass, double $ in PHP..You know it is there you just have to find it.
In a pentest you don’t know which one of the app’s 200 menus are vulnerable. You need a consistent methodology and set of tests to rigourlously assert which common flaws are there, and which are not. You cannot just OCD on one parameter for 2 days looking for a unicode bypass, while a basic SQL injection sits on the 120th parameter..

We can quite easily see that the focus is different. in a CTF you focus more on the exploit rather than the discovery. You quickly spot the SQL injection (there are only 3 menus anyway). The challenge usually is to bypass the black list of forbidden keywords, not use certain special characters, etc.
In a pentest, 99% of SQL injections are trivially exploitable, but you have to find them :). This requires a different skill set and mostly a different flair!

Too much focus on the Web…and Linux

I am bored of Web based challenges springing up like mushrooms on CTF websites. Real life hackers may pwn an app to infiltrate a company if it is a really low hanging fruit, but the real war is happening elsewhere: on email boxes. Do you remember the last hack that did not involve (spear) phishing?

Embedding malware in Office documents, evading spam filters, SPF and DKIM options, executing code with signed applications… This is the juicy stuff to focus on (I wrote a whole chapter on how to do so in How to Hack Like a Pornstar). Which brings me to my next point…

I love Linux, but for fuck sake, nobody cares about it in the business world. So many wargames involving Linux from buffer overflows to obfuscation and polymorphous code, yet it is so useless in pentesting engagements.
In over a decade of pentesting, I think a rooted a Linux server less than 10 times. It was not even mandatory, just for fun on a lazy afternoon. You know what was not optional? Pwning that Windows domain that held emails, network shares full of business documents, accounts with access to HR applications, and of course the CEO’s workstation. That’s where the gold really is.

I tired to illustrate this with my training, where hackers access six real Windows machines and are tasked to pwn the Windows domain to get HR documents. Can’t get any more real than that!

Another type of computers unjustly left behind are Mainframes: 20 Billion transactions per day (credit card, booking, wire, etc.) and still not many security researchers give a damn..That is another battle that deserves its own post.

To sum up

Doing CTFs is great. Have at it! But please keep in mind that there are many other skills to acquire in order to properly conduct a penetration test, let alone hack a company red team style.

Cheers