I hear many people talk about alignment as another problem facing humanity, but we will fix it over time. We've overcome challenging problems in the past, right? Nukes were hard, but we made it through that. We will make it through AI and be fine just the same. We will keep building AI making it more intelligent and figure out alignment as we go. I want to present a model to test this claim. The model dissects technology into two categories: capabilities tech and safety tech.

Capabilities tech consist of tech that makes humans capable of doing more task faster: Bigger rockets, faster cars, and more intelligent AI. Now, we can travel to different places quickly, plant things on the moon, and have text and images generated within seconds. Safety tech, on the other hand, is much different. It keeps the capability tech from backfiring. What precautions are in place to keep humans safe when something inevitably goes wrong? Safety tech keeps the capabilities of tech in check.

Guns firing more bullets faster is capabilities. Making sure the gun doesn't accidentally fire is safety tech. Making a plane fly is capabilities, but adding a parachute is safety. Most technologies we use today are a combination of both of these categories of tech.

Most of the time, safety tech is an afterthought. The first mass-market automobile did not have seat belts or an airbag; those were added later. Security at a tech company is rarely done first. An app is built, and then you make it secure if every. Focusing on capabilities makes economic sense; why would someone waste time making tech safe when they don't know the capabilities? Adding safety normally makes a product take longer to develop, and users don't notice it until it is too late.

This mentality can lead to problems. Imagine that there was some capabilities tech that was so powerful. If it isn't safe, and it messes us, everyone dies. We wouldn't get a second chance to add more safety tech and try again like we do with most other technologies.

I say that AI is just this. It will get smarter and smarter, and unless we do the safety work first, it blows up in our faces catastrophically, and there isn't a redo.

Let me know if you disagree with my framing or have some reason to think that safety tech will be different for AI. I currently don't see enough work being put into the safety tech and much more put into the capabilities tech, and I'm not confident that we will figure out the safety tech in time.