Deep Cogito v2 represents a significant leap forward in the pursuit of artificial general intelligence (AGI), focusing on continual self-improvement and refinement of reasoning abilities. This open AI platform embodies a novel approach to scaling AI intelligence, moving beyond simply increasing model size and computational power. Instead, it emphasizes the internalization of reasoning processes, allowing the AI to develop a stronger "intuition" and improve its problem-solving efficiency.

Iterated Distillation and Amplification (IDA): The Core of Self-Improvement

At the heart of Deep Cogito v2 lies a technique called Iterated Distillation and Amplification (IDA). IDA allows the AI to internalize its own reasoning processes by distilling the discoveries from a search back into the model's core parameters. This process enables the model to anticipate the outcome of its own reasoning without having to perform an entire search, leading to a more direct path to solutions. This approach discourages the model from "meandering" and rewards efficient reasoning.

The Deep Cogito team aims to "hill climb on the gains of iterative self-improvement" in its quest to build superintelligence. They have restated their commitment that all AI models they create will be open-source.

Model Architecture and Performance

The Cogito v2 lineup includes four hybrid reasoning AI models: two mid-sized models at 70B and 109B parameters, and two large-scale versions at 405B and 671B. The largest, a 671B Mixture-of-Experts (MoE) model, is considered one of the most powerful open-source AIs in the world. In reasoning mode, the 671B MoE outperforms DeepSeek R1 and matches the latest DeepSeek R1 0528, using 60% shorter reasoning chains. In non-reasoning mode, it outperforms DeepSeek v3 and matches the performance of the latest DeepSeek v3 0324.

Deep Cogito's open-source AI model can reason about images, a skill they were never explicitly trained for. The team shared an example of this reasoning where Deep Cogito's open-source AI model compared two images of a duck and a lion, demonstrating a deep thinking process about their habitats, colours, and composition purely through transfer learning. Deep Cogito believes this emergent property could be a powerful way to bootstrap training data for future multimodal reasoning systems.

The Significance of Open Source

Deep Cogito's commitment to open source is crucial for fostering collaboration and accelerating progress in AI research. By making its models and techniques publicly available, Deep Cogito enables other researchers and developers to build upon its work, contributing to a more open and transparent AI ecosystem. This approach democratizes access to advanced AI capabilities and encourages innovation across the field.

Continual Learning and Reasoning Abilities

Deep Cogito v2 addresses the critical challenge of continual learning in AI. Continual learning is the ability to pause the model training process, save the model's current state, and then later resume training on new data. The model should be able to generalize well to new data, while still maintaining its ability to generalize to old data. The platform's design allows it to refine its reasoning abilities over time through iterative learning and feedback loops. This is achieved through techniques like Reinforcement Learning from Human Feedback (RLHF), where human preferences guide the model's adjustments. High-quality training data is the backbone of any successful AI model. User feedback plays a pivotal role in enhancing OpenAI's models.

Self-improvement mechanisms allow reasoning LLMs to refine their capabilities over time. Training-Based Self-Improvement involves data collection and refinement strategies, where the model's outputs are used to generate new training examples. Test-Time Self-Improvement is where the model corrects errors and refines its reasoning during the inference process itself.

Implications and Future Directions

Deep Cogito v2 represents a paradigm shift in AI development, moving away from brute-force scaling towards more intelligent and efficient reasoning. Its innovative approach to self-improvement and its commitment to open source position it as a key player in the future of AI. As the platform continues to evolve, it has the potential to unlock new possibilities in various fields, including problem-solving, decision-making, and creative endeavors.

The development of Deep Cogito v2 also highlights the importance of ethical considerations in AI. As AI systems become more advanced, it is crucial to ensure that they are developed and used responsibly, avoiding potential biases and promoting transparency. Deep Cogito's open-source approach can contribute to this goal by allowing for greater scrutiny and collaboration in addressing ethical challenges.

Availability

The Cogito v2 models are available for download on Hugging Face and can be accessed through APIs on platforms like Together AI, Baseten, and RunPod. They can also be run locally with Unsloth.