Teradata launches on-premises AI Factory for secure private AI

Teradata has announced the launch of Teradata AI Factory, an integrated solution delivering the company's cloud-based artificial intelligence (AI) and machine learning (ML) capabilities to secure, on-premises environments.

The AI Factory has been built in collaboration with NVIDIA and unifies key components including data pipelines, algorithm execution, and software infrastructure into a single, scalable system. The solution is intended to accelerate AI development—covering predictive, generative, and agentic AI—through private deployments while facilitating governance, compliance, and security for enterprises.

Teradata AI Factory is designed to integrate software, hardware, and a combination of Teradata and third-party tools, aiming to decrease both compliance risks and costs. When paired with Teradata AI Microservices with NVIDIA and customer-provided NVIDIA GPUs, the platform supports accelerated development, including native Retrieval-Augmented Generation (RAG) pipelines, which are increasingly in demand among data-driven organisations.

The company has positioned this solution as particularly relevant for industries with high regulatory requirements, such as healthcare, finance, and government, as well as any enterprise needing greater control and autonomy over AI strategy and deployments.

Changing requirements

According to the company, current global instability and stricter data sovereignty regulations are influencing organisations to seek more control over their AI infrastructure. These factors coincide with financial pressures that can result from both underused GPU investments and variable cloud computing costs, especially within hybrid enterprise environments. The increasing complexity of AI ecosystems is expected to further drive demand for integrated, turnkey solutions that can address both cost and governance issues.

"Market dynamics are increasing buyer interest in on-premises solutions," said Teradata's Chief Product Officer, Sumeet Arora. "Teradata remains the clear leader in this environment, with proven foundations in what makes AI meaningful and trustworthy: Top-notch speed (performance), predictable cost (resource efficiency), and integration with the golden data record (which may already live on Teradata). Teradata AI Factory builds on these strengths in a single solution for organisations using on-prem infrastructure to gain control, meet sovereignty needs, and accelerate AI ROI."

A recent report from Gartner states: "By 2028, more than 20% of enterprises will run AI workloads (training or inference) locally in their data centers, an increase from approximately 2% as of early 2025." ("How to Determine Infrastructure Requirements for On-Premises Generation AI" by Chandra Mukhyala, Jonathan Forest, Tony Harvey from March 5, 2025)

Feature set

Teradata AI Factory is structured to provide enterprises with a comprehensive on-premises AI solution incorporating security, cost efficiency, and seamless hardware-software integration. Its feature set includes Teradata's Enterprise Vector Store as well as Teradata AI Microservices, the latter of which leverages NVIDIA NeMo microservices to enable native RAG pipeline capabilities.

The platform's architecture aims to address sensitive data requirements by keeping data within the organisation's boundaries, thereby reducing the risks commonly associated with public or shared AI platforms—including data exposure, intellectual property leakage, and challenges with regulatory compliance. Teradata AI Factory supports compliance with established standards such as GDPR and HIPAA, positioning it as an option for organisations where data residency and privacy are priorities.

Its localised set-up is designed to facilitate high levels of AI performance while lowering latency and operational inefficiency due to reduced data movement. Customers can choose to deploy AI models on CPUs or accelerate performance using their existing GPU infrastructure. This approach seeks to avoid unpredictable cloud expenses, allowing organisations to maintain consistent operational costs and prepare for scaled private AI innovation going forward.

Technical integration

Teradata AI Factory presents an integrated, ready-to-run stack for AI applications. It includes:

• AI Platform for Rapid Innovation: Built on Teradata's IntelliFlex platform, the AI Factory incorporates Teradata Enterprise Vector Store, enabling integration of structured and unstructured data for generative AI applications.
• Software Infrastructure: The AI Workbench provides a self-service workspace with access to analytics libraries, including those from ClearScape Analytics. It also offers model lifecycle management, compliance tools, one-click large language model (LLM) deployment, and supports JupyterHub, ModelOps, Airflow, Gitea, and Devpi.
• Algorithm Execution: The system supports scalable execution of predictive and generative algorithms, facilitating high performance through connections with customer GPUs and delivering native RAG processing.
• Data Pipelines: The solution includes data ingestion tools and internal capabilities like QueryGrid, Open Table Format (OTF) compatibility, object store access, and support for NVIDIA utilities for complex data formats such as PDFs.

By processing data locally within an organisation's infrastructure, Teradata AI Factory is intended to enhance data security and operational integrity, providing greater control and certainty for those adopting private AI strategies.