So you want to roll your own crypto?

A coworker of mine asked:

How are people supposed to learn (from mistakes) if they don't roll their own crypto?

The short answer is do roll your own crypto, but don't use it in production until it's vetted by professionals. The long answer below might take a few years to hash out.

Making mistakes is an unavoidable part of the learning process. I've been rolling crypto for Google production for years, but my code is not bug free and will never be. I found that the cheapest way to learn from mistakes is to learn from other people’s mistakes. I recommend taking Cryptography I, doing CTFs, and solving crypto challenges. This won't take long, and very quickly you'd be pretty dangerous because you'd be able to find many crypto bugs.

But it's just the beginning. When I got to Google, I thought I knew crypto because look at all the bugs I found! I was so wrong. It took me years to learn the tradecraft from the real experts which fortunately my employer has plenty.

They say you can become a better programmer by reading good code. Unfortunately, I've learned the hard way that this rule usually does not work in crypto, for 3 reasons:

• Because of side-channel and other constraints, crypto code is usually far from obvious. Imagine coding without the if statement;
• Crypto code usually has many subtle details that look unnecessary, but once changed or removed totally destroy security; and
• Bad crypto code usually looks and produces results indistinguishable from good crypto code.

There's no shortcut rather than learning the fundamentals. It’s not hard, but it takes time. A couple of resources that you might want to check out:

• Cryptography Engineering by Schneier and Ferguson: this was the book that got me started;
• Cryptograph I and the crypto book by Dan Boneh and Victor Shoup: this is hands down the best materials on applied crypto;
• An Introduction to Mathematical Cryptography: if you are worried that you don't know enough math to do crypto, this book will help you put rest to that fear;
• Tink: this is Google’s recommended crypto library (full disclosure: my team owns it). Tink provides high-level APIs, so you won't be able to learn much from it if your goal is to write your own AES, but it has pretty good info on common crypto techniques.

The funny thing is that after spending years studying these resources, you still don’t have a free pass to roll all the crypto in the world. You'd realize and appreciate that crypto is a deep and vast field of study with a very long food chain. Sitting on top are cryptanalysts who

• propose and solve hard mathematical problems such as integer factoring, discrete log, shortest vector, closest vectors, etc; or
• design and break efficient heuristic one-way functions such as AES, ChaCha20, SHA3 or Blake;

and at the bottom are software engineers who want to encrypt some data. Along the chain, you will find people who

• take the one-way functions and build fundamental primitives such as AEAD, MAC, digital signatures, or public key encryption;
• take the primitives and build protocols such as TLS;
• take the protocols and build APIs such as OpenSSL, Bouncy Castle, JCE, Golang Crypto, libsodium or Tink; and
• take the APIs and build applications such as end-to-end encryption, storage encryption, user authentication, etc.

The division is not always clear cut, as there are people who wear multiple hats. The thing is, unless you get to the very top, there are always good reasons not to roll your own crypto. The reasons are usually non-obvious, unknown unknowns. I’ve seen a distinguished engineer encrypting data with AES-ECB arguing that they weren’t rolling their own crypto, because they didn’t implement AES from scratch but called OpenSSL.

So if you want to roll your own crypto, make sure you understand where you are in the crypto food chain and what are the reasons preventing you from moving up. Study and eliminate said reasons. Good luck and have fun!

Discuss this on Hacker News: https://news.ycombinator.com/item?id=24320998.