The Cormackian Challenge and What AI Could Really Do

There were two major stories this week that seem on the surface to have little to do with one another. The first is one you may have come across: Open AI launched Sora 2, their next generation video creation tool that promises to revolutionize the lifelike nature of how moving pictures can be created instantly. The second story is one you may have missed. It's all about the famously reclusive novelist Cormac McCarthy and his vast collection of over 20,000 annotated books.

For as long as AI has been a central topic in the news, we routinely hear about faster and better ways we can generate more content. The bias for using AI to create has become so prevalent that some users describe ChatGPT only as a generative AI tool designed to help you create something new. There's another side of the equation we should consider, and the life and work of Cormac McCarthy is what got me thinking about it.

Right now, there is a team of several people who call themselves "Cormackians" going through every book in McCarthy's library and capturing the annotated notes he made in the margins. Rumored to have a photographic memory and widely admired for his diverse reading habits, some believe that there could be great value in cataloging and taking inspiration from his unpublished ideas. The problem is that this effort is an impossible task:

“If we were a well-funded institution, we’d take all these boxes into an empty building where we had plenty of space to work in, a dedicated team of people and all the time we needed. We can’t be as meticulous as we’d like and scan all the annotations, because we’ve got limited time and a massive amount of books to get through.”

What if there were more people trying to innovate on how to use AI for this purpose? With the recent news that Wikimedia is enabling ways for AI developers to more easily use its data, this idea of AI as a collector and perhaps synthesizer of information rather than solely as a tool for creation is one that deserves more attention. In a world where the sum of human knowledge might still be buried in the margins of a book, we should be looking to technology to try and unlock those insights for us.