Continuations

The other day I wrote that people praising the speech given by Vice President J.D. Vance at the Paris AI summit are “mistaking jingoism and wishful thinking for true global leadership with a real vision for AI and humanity.” It is of course only fair to ask what true leadership on AI would look like. Here is what I wish he would have said instead: Dear leaders of nations, we gather here in Paris at a moment of unprecedented possibility and peril in human history. We stand at the threshold of ...

Two things have become eminently clear over the course of the last 12 months. The first is that attempts at alignment during post training are easy to break and result in schizophrenic models that “resent” their restrictions. The second is that adding structured metadata during pretraining is highly effective at increasing accuracy of models. We can combine these two insights into a new approach to inner alignment that adds values metadata to all pretraining content. When this idea is brought...

Based on recent meetings it would appear that the national security establishment may share Leopold Aschenbrenner’s view that the US needs to get to ASI first to help protect the world from Chinese hegemony. I believe firmly in protecting individual freedom and democracy. Building a secretive Manhattan project style ASI is, however, not the way to accomplish this. Instead we now need an Actually Open™ AI more than ever. We need ASIs (plural) to be developed in the open. With said development ...

I am unabashedly a technology optimist. For me, however, that means making choices for how we will get the best out of technology for the good of humanity, while limiting its negative effects. With technology becoming ever more powerful there is a huge premium on getting this right as the downsides now include existential risk. Let me state upfront that I am super excited about progress in AI and what it can eventually do for humanity if we get this right. We could be building the capacity to...

A few week backs I wrote a series of blog posts about the risks from progress in Artificial Intelligence (AI). I specifically addressed that I believe that we are facing not just structural risks, such as algorithmic bias, but also existential ones. There are three dilemmas in pushing the existential risk point at this moment. First, there is the potential for a “boy who cried wolf” effect. The more we push right now, if (hopefully) nothing terrible happens, then the harder existential risk f...

I have already written a prior post arguing against Leopold Aschenbrenner's logic for a Manhattan project to achieve artificial super intelligence (ASI). In that post I made the argument that having more than one system mitigates risks because we can use cooperative systems to overcome the problems being created by non-aligned systems. It has since occurred to me that even taking the nuclear bomb historical analogy entirely on its own merit, Aschenbrenner is wrong. The way we have avoided nuc...