Define the Components of Your AI Architecture

Set up the foundation of your AI architecture for success.

Analyst Perspective

The rapid pace of AI innovation necessitates building systems that can adapt and evolve over time.

Irina Sedenko

Technical Counselor
Info-Tech Research Group

Ibrahim Abdel-Kader

Senior Research Analyst
Info-Tech Research Group

Rapid technological advancements have increased the complexity of AI systems. But these advancements have also enabled new use cases and applications of AI with the introduction of new AI frameworks, design patterns, and model types used by systems. To leverage those new opportunities, it is important to continuously evolve best practices for designing AI systems.

When done correctly, AI architecture is not a monolithic technology stack that requires major modifications for a new implementation, but rather a flexible and scalable platform that allows seamless integration of the components required for a new system or a new business use case.

Understanding the building blocks and the dependencies of the components involved is crucial, since they form the backbone of every AI platform implementation. Do not blindly chase the hype! Learn what it takes to build solutions properly, so you don’t get stuck with costly fixes that act as temporarily pain relief.

Review considerations before buying or building solutions from scratch. Every decision matters! Our research and experts will help you navigate the increasing complexities and gain the confidence to scale the effectiveness of AI architectures powered by systems and models that require these architectures to handle computationally intensive AI applications and massive data sets and run real-time inference.

Executive Summary

Info-Tech Insight

Build your target state architecture from predefined best-practice building blocks. An AI platform is not a monolithic technology stack that requires major modifications for a new implementation, but rather a flexible and scalable platform that allows seamless integration of the components required for new use cases.

Key Concepts

AI Lifecycle

The AI lifecycle includes all events and processes that relate to an AI system’s life span from inception to decommissioning, including its design, research, training, development, deployment integration, operation, maintenance, sale, use, and governance.1

Data Sources

Data sources are the origins of the data used for analysis, modeling, and decision-making in AI solutions. They can include structured data from databases, unstructured data from text documents, and real-time data from sensors and IoT devices.

Data Engineering Engine

A data engineering engine automates the processes of collecting, transforming, and preparing data for analysis and machine learning, ensuring that data is clean, reliable, and ready for use in AI models. It integrates various data sources and applies data processing techniques to support the development and deployment of AI solutions.

Data Science and Model Development

Data science includes a broad grouping of mathematics, statistics, probability, computing, and data visualization used to extract knowledge from a heterogeneous set of data (images, sound, text, genomic data, social network links, physical measurements, etc.).2 Model development consists of processes to ensure that various principles are met and ready for deployment in accordance with AI solution objectives.

Infrastructure

Infrastructure includes foundational physical and virtual resources, such as servers, storage, networking, and software, that are necessary to support the development, deployment, and operation of AI solutions. It ensures that the AI system has the necessary capabilities to function effectively and efficiently.

Data Platform

A data platform supports the collection, storage, processing, and analysis of data, enabling organizations to manage and use their data assets effectively. It integrates various data sources and tools to provide a unified environment for data-driven decision-making.

Sources: 1. GOV.UK, 2024; 2. Council of Europe, 2021

AI fails to deliver success

Avoid contributing to this trend.

80%

… of AI projects fail, which is twice the failure rate of non-AI-related start-ups.

60%

… of AI projects fail due to misalignment of goals between key stakeholders and the technical team.

55%

… of AI projects fail due to poorly prepared data sets.

50%

… of models are delivered of what they could have been due to leaders directing the team to move on from projects prematurely.

14%

… of organizations are fully ready to integrate AI into their businesses.

Source: RAND, 2024

Launch your AI proof of concept

“We need to launch an AI proof of value to assess the viability of the technology before we invest.”

Leaders need to show that AI initiatives can work as intended and will yield tangible business benefits.

Assessments face significant time constraints.

If the project team takes too long to secure support, this could lead to funding delays or missed opportunities.

This research is focused on these steps.

Adopt a formal methodology to accelerate AI vendor selection and improve outcomes.

Identify relevant capabilities to support the AI use cases you identified.

Validate AI use cases against business requirements.

Identify vendors to support AI use cases, devise a vendor selection model, and select products for review.

Every successful AI architecture goes through a journey

AI Lifecycle Stages: Potential AI Risks and Activities

See Info-Tech’s Govern the Use of AI Responsibly With a Fit-for-Purpose Structure for more information on AI risks

Source: Based on National Institute of Standards and Technology (NIST), 2023

It’s imperative to understand the building blocks of your AI architecture

AI Reference Architecture: High Level

Select the right components to fully optimize your AI architecture

AI Reference Architecture: Detailed

Choose the AI solution path that will lead to success

Consider these phases as you buy and configure your AI architecture

AI Lifecycle and Reference Architecture Alignment – Buy and Configure

Consider these phases as you build your AI architecture

AI Lifecycle and Reference Architecture Alignment – Build

Define the Components of Your AI Architecture: Project Overview

Define the Components of Your AI Architecture

1. Plan and design AI use cases.
2. Select AI components.
3. Draft your reference architecture.

Guided Implementations

Call 1 – Identify your goals and objectives.

Call 2 – Document AI use cases.

Call 3 – Set a solution path.

Call 4 – Identify stakeholders.

Call 5 – Review the considerations for each AI component.

Call 6 – Select the AI components of your architecture.

Call 7 – Draft a reference architecture.

Call 8 – Document high-level workflows and implications.

Outcome

• AI business goals and objectives.
• Value proposition and expected outcomes of each AI use case.
• Decision to buy capabilities to support each AI use case, build new capabilities from scratch, or extend existing capabilities to support the AI use case.
• Members of the AI project team.

Decision on components that enable AI architecture by:

• Further defining each AI use case with architecture considerations.
• Selecting the components for the AI architecture.

Overview of how AI reference architecture and use case summaries lead to successful organization outcomes, including:

• Initial AI reference architecture.
• Documentation of high-level workflows and implications.

Insight Summary

Overarching Insight

Build your target state architecture from predefined best-practice building blocks. An AI platform is not a monolithic technology stack that requires major modifications for a new implementation, but rather a flexible and scalable platform that allows seamless integration of the components require cases.

Use Case Insight

Not all business use cases require AI to improve business capabilities.

Determine how the AI use case will generate business value before you commit to a full-scale implementation.

Solution Path Insight

Knowing the AI solution path that you want to use will simplify the architecture considerations.

Building Blocks Insight

Recognizing the fundamental building blocks and their interdependencies is essential for any AI platform implementation.

By mastering these core elements, you can effectively design any type of AI architecture and prevent costly mistakes.

Tactical Insight

To achieve many successful AI implementations, plan your target state architecture with clear milestones and deliver it in phases, implement robust model versioning and deployment management practices, and continuously monitor the AI system in production using key performance indicators (KPIs) such as model accuracy, deployment success rates, and system uptime.

Blueprint Deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals.

AI Architecture Component Selection Tool

Use this tool to help support the selection of AI components associated with the architecture design.

Key Deliverable

AI Architecture Executive Presentation Template

Use this template to help you build a clear and compelling case for an AI architecture design that you can present to executive management and sponsors.

Benefits of a component-based approach

IT Benefits

• Simplifies the process of architecture creation, since it provides a layered approach to hide or expose details
• Provides a modeling framework that allows you to describe, visualize, and analyze your business structure in a defined and controlled manner
• Gives a roadmap earmarked by relatively stable states when certain capabilities can be implemented
• Includes research on standard AI architecture patterns, which reduces the need for custom research

Business Benefits

• Identifies required business capabilities that will be fulfilled by AI
• Validates business capabilities that provide information on the return and viability of given current architecture
• Provides insight on the roles, responsibilities, and timelines for different groups developing the AI-based capabilities
• Saves time by building AI architecture from tried and tested architecture building blocks
• Gives sooner insight into the success of an AI project

Measure the value of this blueprint

• Increase the number of entities in the platform that support AI.
• Measure progress by the attainment of architecture plateaus as defined in your roadmap.
• Measure the time that data scientists have to spend wrangling data.

The more success indicators that are met, the better equipped your AI program is to scale the architecture and handle complexity.

Establish Baseline Metrics

Baseline metrics will be improved through:

1. Improving AI implementation and AI lifecycle practices.
2. Planning the target state architecture and delivering it in phases.
3. Putting management practices in place for model versioning and model/system deployment.
4. Monitoring the AI system in production.

Phase 1

Define the Components of Your AI Architecture

Phase 1

1.1 Plan and design AI use cases.

1.2 Select AI components.

1.3 Draft your reference architecture.

This phase will walk you through the following activities:

• Identify goals and objectives.
• Document AI use cases.
• Set a solution path.
• Identify stakeholders.
• Review the considerations for each AI component.
• Select the components of your AI architecture.
• Draft your reference architecture.
• Document high-level workflows and implications.

Define the Components of Your AI Architecture

This phase involves the following participants:

• IT leader
• Technical IT lead
• Business analyst
• Business lead
• Process expert