The Big Idea: Bruce Schneier

The world has systems. Systems have rules. Or are they more like guidelines? In today’s Big Idea for A Hacker’s Mind, security expert Bruce Schneier takes a look at systems, how they are vulnerable, and what that fact means for all of us.

BRUCE SCHNEIER:

Hacking isn’t limited to computer systems, or even technology. Any system can be hacked.

What sorts of system? Any system of rules, really.

Think about the tax code. It’s not computer code, but it’s a series of rules — supposedly deterministic algorithms — that take data about your income and determine the amount of money you owe. This code has vulnerabilities, more commonly known as loopholes. It has exploits; those are tax avoidance strategies. And there is an entire industry of black-hat hackers who exploit vulnerabilities in the tax code: we call them accountants and tax attorneys.

In general terms, a hack is something a system permits, but that is unanticipated and unwanted by its designers. It’s unplanned: a mistake in the system’s design or coding. It’s clever. It’s a subversion, or an exploitation. It’s a cheat ­- but only sort of. Just as a computer vulnerability can be exploited over the Internet because the code permits it, a tax loophole is “allowed” by the system because it follows the rules, even though it might subvert the intent of those rules.

Once you start thinking of hacking in this way, you’ll start seeing hacks everywhere. You can find hacks in customer reward programs; in financial systems; in politics; in lots of economic, political, and social systems; and against our cognitive functions. Airline frequent-flier mileage runs are a hack. The filibuster was originally a hack, invented in 60 BCE by Cato the Younger, a Roman senator. Gerrymandering is a hack. Hedge funds are full of hacks. So are professional sports: curving a hockey stick, hitting a cricket ball over your head, or showing up on the Formula One track with a six-wheeled car (the Tyrell racing team in 1975 — really).

I use this framework in A Hacker’s Mind to tease out a lot of why today’s economic, political, and social systems are failing us so badly, and apply what we have learned about hacking defenses in the computer world to those more general hacks. There’s a lot of value in looking at these systems through the lens of hacking.

All systems are hackable. Even the best-thought-out sets of rules will be incomplete or inconsistent. They’ll have ambiguities, and things the designers haven’t thought of. As long as there are people who want to subvert the goals of a system, there will be hacks.

What will change everything is artificial intelligence, and what will happen when AIs start hacking. Not the problems of hacking AI, which are both ubiquitous and super weird, but what happens when an AI is able to discover new hacks against these more general systems. What happens when AIs find tax loopholes, or loopholes in financial regulations. We have systems in place to deal with these sorts of hacks, but they were invented when hackers were human and reflect the human pace of hack discovery. They won’t be able to withstand an AI finding dozens, or hundreds, of loopholes in the financial network. We’re simply not ready for the speed, scale, scope, and sophistication of AI hackers.

Hacks aren’t necessarily bad. They’re how systems evolve. Curved hockey sticks made for more exciting play, as did scooping a cricket pitch — they both became part of the games. A six-wheeled race car was declared against the rules in 1983. Mileage runs are legal, but airlines have modified their frequent-flier programs to make them less effective. Gerrymandering is still mostly legal in the US, and the filibuster is still a thing in the US Senate.

A Hacker’s Mind is my pandemic book, started in 2020 and finished in 2022 It represents another step in my continuing journey in thinking about security and its relationship to broader society.) And I really like the cover.

A Hacker’s Mind: Amazon|Barnes & Noble|Bookshop|Powell’s

Visit the author’s site. Follow him on Twitter.