Undoubtedly DeepSeek is introducing a new era for AI, highlighting that different paths might be followed in implementing effective AI infrastructure and optimise the related costs.
DeepSeek, with its R1 model, diverges significantly from traditional AI structures like those powered by NVIDIA, both in terms of operational architecture and resource efficiency.
Here are the main structural differences:
Mixture of experts (MoE) architecture: DeepSeek R1 uses an MoE approach, allowing for selective parameter activation (only 37 billion out of 671 billion) based on the task at hand. This contrasts with NVIDIA’s models like o1, which often rely on a fully engaged network for every query, leading to higher computational demands.
Dynamic inference: DeepSeek R1’s model can scale its computational effort according to the complexity of the problem, enhancing efficiency for both simple and complex tasks. NVIDIA’s models typically operate at full capacity regardless of task complexity, which can be resource-intensive.
Mixed precision computing: DeepSeek R1 employs a strategy where it uses both 8-bit and 32-bit precision, enabling faster processing with minimal accuracy loss. This is less common in traditional models, which might stick to higher precision across all operations, thus consuming more resources.
DeepSeek versus other models: Resource utilisation efficiencies
The new structure leads to considerable resource utilisation efficiencies, including:
- GPU optimisation: DeepSeek R1 was developed using fewer, less powerful GPUs, making it more accessible for firms with constrained resources. NVIDIA’s solutions often require high-end GPUs in large quantities, escalating costs. Reports indicate that DeepSeek’s R1 model was developed using approximately 2,000 Nvidia H800 GPUs, significantly fewer than the tens of thousands typically employed by competitors, resulting in considerable cost savings. That leads to the fact that AI models can be developed using GPUs that must not be necessarily sourced from the latest state-of-the-art Nvidia products.
- Lower operational costs: The efficiency of DeepSeek R1 means financial institutions can deploy AI at scale with significantly reduced costs, a key consideration in an industry where margins are often tight.
Cost-benefit analysis for financial institutions
The financial sector stands to gain significantly from AI models that deliver robust performance without incurring prohibitive costs.
DeepSeek‘s R1 model exemplifies this balance by offering high-level capabilities at a fraction of the traditional expense. The company has demonstrated that its AI models can be developed with less advanced hardware, resulting in considerable cost savings. DeepSeek R1’s development cost was around $5.58m, a fraction compared to the billions required for NVIDIA’s top-tier models. This cost efficiency can be a game-changer for financial firms looking to implement AI without prohibitive expenses.
Furthermore, the model’s architecture allows for scaling AI operations without a linear increase in cost, enabling firms to handle increased volumes of data analysis or decision-making during peak market times.
For financial institutions, this translates to the ability to implement advanced AI-driven analytics and decision-making tools without the need for extensive capital investment in infrastructure. The reduced energy consumption further contributes to operational savings and aligns with growing environmental, social, and governance (ESG) considerations.
However, it’s essential to recognise that while DeepSeek’s models offer cost advantages, they may not yet match the performance of NVIDIA-powered solutions in all scenarios. NVIDIA’s hardware and software ecosystems are deeply entrenched in the AI industry, providing optimised performance for a wide range of applications. Financial institutions must carefully assess their specific needs, evaluating whether the cost savings with DeepSeek’s models justify any potential trade-offs in performance or compatibility.
Risks of AI dependency in financial institutions
Despite the allure of advanced AI models, financial institutions must exercise caution to avoid overdependence. An overreliance on AI can lead to several risks:
- Systemic risk: Over-reliance on AI, even with models like DeepSeek R1, can introduce systemic risks. If AI systems fail or are manipulated, the consequences could ripple through financial markets, an issue I’ve often highlighted in discussions on financial stability.
- Model risk: All AI models, including DeepSeek, operate as “black boxes”, making it challenging to interpret decision-making processes and are susceptible to manipulation or ‘jailbreaking’. There’s a particular risk with DeepSeek R1 due to its open-source nature, where malicious actors could exploit known vulnerabilities or manipulate input to skew outputs, leading to flawed financial decisions or security breaches.
- Manipulation of outputs: Deep manipulation of AI outputs is a universal concern, but with DeepSeek R1, this risk is heightened due to its broad accessibility. In finance, where decisions can move millions, ensuring the integrity of AI outputs is paramount. The potential for adversaries to craft inputs that lead to desired but incorrect outputs (like in adversarial attacks) poses a significant threat.
- Data quality and bias: AI systems are only as effective as the data they are trained on. Poor-quality or biased data can result in inaccurate predictions or reinforce existing biases, leading to flawed decision-making. While the R1 model has shown a great advantage in training costs, the quality of such analysis is still linked to the quality and depth of data it is fed with.
- Regulatory and ethical compliance: As AI becomes more integrated into financial decision-making, regulatory eyes sharpen. The open-source aspect of DeepSeek could complicate compliance with data privacy laws and ethical AI use policies.
- Operational continuity: An over-dependence on AI could disrupt operations if systems go down or if the AI’s decision-making is compromised. Financial institutions need robust backup systems and human oversight to mitigate this.
- Human oversight reduction: There’s a risk that the reliance on AI might diminish the role of human judgement, which is crucial for ethical decision-making and nuanced risk assessment, areas where AI can be lacking.
In conclusion, while DeepSeek R1 offers compelling advantages in terms of cost and efficiency, the integration into financial services must be tempered with caution. The benefits of optimised AI-driven analytics are clear, but the risks, particularly around manipulation and dependency, require vigilant risk management.
A hybrid model where AI supports but does not replace human expertise seems to be preferable, especially in the complex world of finance where every decision carries weight. Ensuring AI models are part of a broader, secure, and ethical framework is essential to harnessing their power responsibly.
The writer is the CEO of Axiory Global.
Read: AI-powered desktop for $3,000? This is Nvidia’s plan for Project DIGITS
