By Tyler Saltsman - February 5, 2025

Disclaimer: These are my personal views and do not necessarily reflect the opinions of my team. We foster a culture of open debate, direct communication, and greatly value diversity of thought.

We’ve entered a pivotal moment in the evolution of AI—a transformative era that, while marked by incredible progress, also presents profound questions about what comes next. The established scaling laws that once governed the pre-training of models have been disrupted. The old reliance on sheer compute power, once thought to be key to advancing AI capabilities, is showing diminishing returns. Rather, we are coming to the realization that no amount of compute can truly make up for a lack of high-quality, well-curated data and domain expertise. This is something many in the AI space, including the giants like OpenAI, perhaps overestimated in the past. The true breakthrough in this new era isn’t just about raw power—it’s about the smart application of reinforcement learning, a feat of exceptional engineering.

What’s striking is that with just 8 billion parameters—arguably even with as few as 3 billion—we’re achieving performance levels that rival GPT-4. That would have been almost inconceivable to anyone in the field two years ago. If I were in the shoes of companies like Anthropic, Mistral, or OpenAI, I would be deeply worried about my competitive moat. The notion that "frontier models" belong to only a few well-funded companies is quickly being upended. We are seeing the start of “perfect competition" and a race to zero in terms of inference costs. The open-source community is not only catching up, but also surpassing the performance of proprietary models. This is exactly the kind of disruption we predicted from the start.

Looking forward, DeepSeek’s models, though impressive, are still not ideal for heavily regulated industries and national security applications due to issues like bias—especially biases introduced by data contamination and poison. Nevertheless, there are invaluable lessons we can take from DeekSeek’s model architecture and training efficiencies. We can build on these learnings and apply them to models that are not only increasingly efficient, but also trained on fully transparent and auditable data. Their team has set the stage for the next wave of innovation.

To continue on this trajectory, we must seek out and integrate the brightest minds in mathematics—particularly those from fields like high-frequency trading. These are the individuals who know how to transform abstract theory and turn raw mathematical concepts into kernels—functions that transform data into higher dimensions, revealing hidden patterns. This is why one of our first research hires was a math PhD—because we need that depth of understanding to unlock the potential of AI.

However, even with all the progress we’ve made, it’s essential to acknowledge that we’re still a long way from true AI reasoning. I say this not to sound overly critical, but because I believe it’s a crucial distinction. Right now, these models are simulating reasoning, but that’s not the same as authentic, conscious reasoning. Interestingly, a lot of human behavior isn’t rooted in true reasoning either; much of our actions are driven by instinct and past experience, operating on pattern recognition—just like how language models predict the next token in a sequence.

What does true AI reasoning look like and how do we recognize it? It starts with a solid foundation of "truth"—understanding what is right and wrong, good and evil, in a way that transcends mere pattern recognition. This is where structured data of knowledge graphs, and well-defined ontologies come into play. Once we establish this framework, AI will be able to reason more authentically, marking the next true leap in its development. Building AI is like creating a work of art—we shape it in our own image. However, this can be deeply problematic, given how flawed we are as humans.

But that "truth" needs to be grounded in something meaningful—something that resonates with the very core of our shared human experience. And here, I believe we come back to values. In many ways, the stability of democratic norms rests on a set of shared values that have evolved over centuries—values born from the ancient Greeks and shaped and refined through the lens of Enlightenment thinking. These principles—justice, fairness, and human dignity—are what bind us together, regardless of individual beliefs. Replacing these foundational principles with something like AGI could have catastrophic consequences, as history has repeatedly shown. Ideologies that discard shared truths in favor of a single, all-powerful authority—whether it’s a totalitarian regime like Stalin’s communist atheism or Mao Zedong’s Maoism, or even worse, an omnipotent AGI that humans worship and turn to for guidance—have historically led to disastrous outcomes, and there's every reason to believe they would do so again.

Sam Altman’s vision, for all its ambition, carries immense risk. His approach, in some respects, seems to move toward replacing these foundational values with the emergence of AGI—a powerful, uncontrollable force. This could not only endanger national security, but also threaten the very principles that have guided us for millennia. While my own personal values are irrelevant, I deeply believe in the values that have been woven into the fabric of our society, which is why we are the greatest nation on earth. These values, though not universally agreed upon, provide a moral compass that informs our actions and decisions. As AI continues to advance, it is crucial that these values remain embedded within the technology. Without them, we risk losing the essence of what makes us human.

DeepSeek is a legitimate issue and has caused significant disruption, but this chaos can actually be seen in a positive light. DeepSeek’s models, for example, are already outpacing even Anthropic’s latest offerings, all while running at a fraction of the cost. They have mastered the Mixture of Experts (MoE) architecture, putting them ahead of the competition in terms of both efficiency and capability. Furthermore, DeepSeek’s self-hosting ability gives it a significant edge over other frontier model providers. The real breakthrough isn’t about simply scaling up hardware—it's about leveraging smarter, more effective methodologies to extract value from data.

China remains a formidable competitor. Their national emphasis on mathematics has cultivated a vast pool of brilliant minds who excel at turning raw mathematical concepts into efficient, high-performance models. But where DeepSeek truly distinguishes itself is in its breakthroughs with reinforcement learning, which have made traditional scaling laws—once relied upon by companies like OpenAI—largely obsolete. We’re entering a new era where the true leaders in AI will be those who can combine deep theoretical knowledge with practical application. And in this race, the stakes are incredibly high.

While DeepSeek’s models have their imperfections—particularly in terms of bias—they are already challenging and, in many cases, surpassing the best in the field. This is an exciting time, and the competition has never been more intense. We must learn from them just as they have learned from us. More importantly though, we must understand the training data and the bias since our warfighters cannot build systems on top of models that do not share our values.

One note on DeepSeek’s reported training costs: it’s broadly agreed by the research community that the $5.5M figure is an understatement. That number likely represents the cost of a single training run, but the full cost is likely closer to $50M or more.

DeepSeek also used a distillation process, refining a much larger model into a smaller, more efficient one—requiring far more compute than is initially apparent for R1. But the key takeaway is that DeepSeek is achieving an order of magnitude more efficiency and they have open sourced their models, making them a serious player on the world stage.

The distillation process they’ve perfected—originally pioneered by Google with their BERT model—enables them to bypass traditional methods that many in the AI community once considered essential. This innovation has allowed DeepSeek’s models to be not only more cost-effective but also far more efficient, positioning them as a powerful force in this fiercely competitive field. How they managed to make the "student outshine the master" through distillation is truly mind-boggling and warrants study.

This is a critical moment for the future of AI. The next few years will determine which models—and which values—will define the future of the field. While LLMs have come a long way, they still struggle with tasks involving time-series data and physics. A significant issue with these models is their inability to truly understand the physical world. But that will soon change. With the advent of Vision-Language Models (VLMs), such as those being integrated into drones, vehicles, and robots, we’ll see a new wave of synthetic data that will enable models to better understand the physical world.

Describing the Sistine Chapel is one thing, but standing in its presence, smelling the paint, and feeling the space is something entirely different. Similarly, we can study war to learn about it, but experiencing it firsthand on the battlefield is a whole other level. As VLMs become ubiquitous, this new understanding of the world will usher in a new era of AI—a transformation driven by richer, more nuanced data routing back to the datacenter (near edge).

Moreover, this isn’t just a technological race; it’s a matter of national security. We cannot afford to fall behind in the AI race to China like we did with drones. The brightest minds in the U.S. must unite, much like during the Manhattan Project, to ensure that we are at the forefront of this pivotal moment in history. AGI will eventually arrive—and that’s a terrifying prospect. But if we can reach it first as a unified country, we may be able to contain, control, and guide it. Whether we can achieve that is uncertain, which is why I don’t believe in the pursuit of AGI. Instead, I believe in a collective swarm intelligence—a network of millions of AI agents working together to augment, rather than replace, human intelligence.

I’m not particularly religious, and my views on religion are personal, but I deeply value the principles that have shaped core democratic principles and modern human rights. These principles—justice, duty, honor, compassion, a warrior ethos as a forcing function for good—are not just abstract ideas; they are the foundation of our culture as Americans. In my time in the military, we often said prayers before missions, and those moments reminded us of the higher ideals that guided our actions. We would also provide aid to the enemy wounded even at our own peril because it is the right thing to do. Why? Because we’re Americans. AGI, however, would not operate with this kind of moral compass. It would lack the compassion, the ethical responsibility, the deep sense of right and wrong that makes us who we are.

As we continue to develop AI, we must ensure that it reflects the best of our values–an honor code. It must preserve the soul of humanity—acting not just logically, but ethically. 

If we lose sight of that, we risk losing something much more profound than just technological dominance—we risk losing our very essence as human beings.