- Understanding the impact of the machine learning/AI component that is built into most of the enterprise products and tools and its role in the implementation of the solution.
- Understanding the most important aspects that the organization needs to consider while planning the implementation of the AI-powered product.
Our Advice
Critical Insight
- Organizations are faced with multiple challenges trying to adopt AI solutions. Challenges include data issues, ethics and compliance considerations, business process challenges, and misaligned leadership goals.
- When choosing the right product to meet business needs, organizations need to know what questions to ask vendors to ensure they fully understand the implications of buying an AI/ML product.
- To guarantee the success of your off-the-shelf AI implementation and ensure it delivers value, you must start with a clear definition of the business case and an understanding of your data.
Impact and Result
To guarantee success of the off-the-shelf AI implementation and deliver value, in addition to formulating a clear definition of the business case and understanding of data, organizations should also:
- Know what questions to ask vendors while evaluating AI-powered products.
- Measure the impact of the project on business and IT processes.
Drive Business Value With Off-the-Shelf AI
A practical guide to ensure return on your Off-the-Shelf AI investment
Executive Summary
Your Challenge
|
Common Obstacles
|
Info-Tech’s Approach
Info-Tech’s approach includes a framework that will guide organizations through the process of the Off-the-Shelf AI product selection. To guarantee success of the Off-the-Shelf AI implementation and deliver value, organization should start with clear definition of the business case and an understanding of data. Other steps include:
|
Info-Tech Insight
To guarantee the success of your Off-the-Shelf AI implementation and ensure it delivers value, you must start with a clear definition of the business case and an understanding of your data.
Info-Tech offers various levels of support to best suit your needs
DIY Toolkit |
Guided Implementation |
Workshop |
Consulting |
"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful." | "Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track." | "We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place." | "Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project." |
Diagnostics and consistent frameworks used throughout all four options
Getting value out of AI and machine learning investments
92.1%of companies say they are achieving returns on their data and AI investments |
91.7%said they were increasing investments in data and AI |
26.0%of companies have AI systems in widespread production |
However, CIO Magazine identified nine main hurdles to AI adoption based on the survey results:
|
“Data and AI initiatives are becoming well established, investments are paying off, and companies are getting more economic value from AI.” (Source: NewVantage, 2022.)
“67% of companies are currently using machine learning, and 97% are using or planning to use it in the next year.” (Source: Deloitte, 2020) |
AI vs. ML
Machine learning systems learn from experience and without explicit instructions. They learn patterns from data then analyze and make predictions based on past behavior and the patterns learned. Artificial intelligence is a combination of technologies and can include machine learning. AI systems perform tasks mimicking human intelligence such as learning from experience and problem solving. Most importantly, AI is making its own decisions without human intervention. The AI system can make assumptions, test these assumptions, and learn from the results.
|
“Machine learning is the study of computer algorithms that improve automatically through experience.” (Tom Mitchell, 1997) “At its simplest form, artificial intelligence is a field, which combines computer science and robust datasets, to enable problem-solving.” (IBM, “What is artificial intelligence?”) |
Types of Off-the-Shelf AI products and solutions
ML/AI-Powered Products | Off-the-Shelf Pre-built and Pre-trained AI/ML Models |
|
|
Examples of OTS tools/products:
|
Examples of OTS models:
|
The data inputs for these models are defined, the developer has to conform to the provided schema, and the data outputs are usually fixed due to the particular task the OTS model is built to solve.
Insight summary
Overarching insight:
To guarantee the success of your Off-the-Shelf AI implementation and ensure it delivers value, you must start with a clear definition of the business case and an understanding of your data.
Business Goals
Question the value that AI adds to the tool you are evaluating. Don’t go after the tool simply because it has an AI label attached to it. AI/ML capabilities might add little value but increase implementation complexity. Define the problem you are solving and document business requirements for the tool or a model.
Data
Know your data. Determine data requirements to:
- Train the model during the implementation and development.
- Run the model in production.
People/Skills
Define the skills required for the implementation and assemble the team that will support the project from requirements to deployment and support, through its entire lifecycle. Don’t forget about production support and maintenance.
Choosing an AI-Powered Tool
No need to reinvent the wheel and build a product you can buy, but be prepared to work around tool limitations, and make sure you understand the data and the model the tool is built on.
Choosing an AI/ML Model
Using Off-the-Shelf-AI models enables an agile approach to system development. Faster POC and validation of ideas and approaches, but the model might not be customizable for your requirements.
Guaranteeing Off-the-Shelf AI Implementation Success
Info-Tech Insight
To guarantee the success of your Off-the-Shelf AI implementation and ensure it delivers value, you must start with a clear definition of the business case and an understanding of your data.
Why do you need AI in your toolset?Business GoalsClearly defined problem statement and business requirements for the tool or a model will help you select the right solution that will deliver business value even if it does not have all the latest bells and whistles. |
![]() |
Do you know the data required for implementation?DataExpected business outcome defines data requirements for implementation. Do you have the right data required to train and run the model? |
![]() |
Is your organization ready for AI?People/Team/ SkillsNew skills and expertise are required through all phases of the implementation: design, build, deployment, support, and maintenance, as well as post-production support, scaling, and adoption. Data Architecture/ InfrastructureNew tool or model will impact your cloud and integration strategy. It will have to integrate with the existing infrastructure, in the cloud or on prem. |
![]() |
What questions do you need to ask when choosing the solution?Product/ Tool or Model SelectionDo you know what model powers the AI tool? What data was used to train the tool and what data is required to run it? Ask the right questions. |
![]() |
Are you measuring impact on your processes?Business and IT ProcessesBusiness processes need to be defined or updated to incorporate the output of the tool back into the business processes to deliver value. IT governance and support processes need to accommodate the new AI-powered tool. |
![]() |
Realize and measure business value of your AI investmentValueDo you have a clear understanding of the value that AI will bring to your organization?Optimization?Increased revenue?Operational efficiency? |
Introduction of Off-the-Shelf AI Requires a Strategic Approach
Business Goals and Value | Data | People/Team/ Skills | Infrastructure | Business and IT Processes | |
AI/ML–powered tools |
|
|
|
|
|
Off-the-shelf AI/ML pre-built models |
|
|
|
|
|
Define Business Goals and Objectives
Why do you need AI in your toolset? What value will AI deliver? Have a clear understanding of business benefits and the value AI delivers through the tool.
Info-Tech InsightQuestion the value that AI adds to the tool you are evaluating. Don’t go after the tool simply because it has an AI label attached to it. AI/ML capabilities might add little value but increase implementation complexity. Define the problem you are solving and document business requirements for the tool or a model. | ![]() AAI solutions and technologies are helping organizations make faster decisions and predict future outcomes such as:
|
1. Use Info-Tech’s Off-the-Shelf AI Analysis Tool to define business drivers and document business requirements
2-3 hours![]() |
Use the Business Drivers tab to document:
|
|
|
![]() |
Download the Off-the-Shelf AI Analysis Tool
1. Define business drivers and document business requirements
Input
|
Output
|
Materials
|
Participants
|
Understand data required for implementationDo you have the right data to implement and run the AI-powered tool or AI/ML model? | Info-Tech InsightKnow your data. Determine data requirements to:
| ||||||||
Availability | ![]() | Quality | ![]() | Preparation | ![]() | Bias, Privacy, Security | ![]() | Data Architecture | |
|
|
|
|
|
2. Use Info-Tech’s Off-the-Shelf AI Analysis Tool to document data requirements
2-3 hours
Use the Data tab to document the following for each data source or dataset:
|
![]()
Download the Off-the-Shelf AI Analysis Tool |
2. Document data requirements
Input
| Output
| Materials
| Participants
|
Is Your Organization Ready for AI?
Assess organizational readiness and define stakeholders impacted by the implementation. Build the team with the right skillset to drive the solution.
|
|
Info-Tech Insight:
Define the skills required for the implementation and assemble the team that will support the project through its entire lifecycle. Don’t forget about production, support, and maintenance.
3. Build your implementation team
1-2 hours
Input: Solution conceptual design, Current resource availability
Output: Roles required for the implementation of the solution, Resources gap analysis, Training and hiring plan
Materials: Whiteboard/Flip charts, Off-the-Shelf AI Analysis Tool, “People and Team” tab
Participants: Project lead, HR, Enterprise Architect
- Review your solution conceptual design and define implementation team roles.
- Document requirements for each role.
- Review current org chart and job descriptions and identify skillset gaps. Draft an action plan to fill in the roles.
- Use Info-Tech’s Off-the-Shelf AI Analysis Tool's People and Team tab to document team roles for the entire implementation, including design, build/implement, deployment, support and maintenance, and future development.
Download the Off-the-Shelf AI Analysis Tool
Cloud, SaaS or On Prem – what are my options and what is the impact?
Depending on the architecture of the solution, define the impact on the current infrastructure, including system integration, AI/ML pipeline deployment, maintenance, and data storage
- Data Architecture: use the current data architecture to design the architecture for an AI-powered solution. Assess changes to the data architecture with the introduction of a new tool to make sure it is scalable enough to support the change.
- Define infrastructure requirements for either Cloud, Software-as-a-Service, or on-prem deployment of a tool or model.
- Define how the tool will be integrated with existing systems and into existing infrastructure.
- Define requirements for:
- Data migration and data storage
- Security
- AI/ML pipeline deployment, production monitoring, and maintenance
- Define requirements for operation and maintenance of the tool or model.
- Work with your infrastructure architect and vendor to determine the cost of deploying and running the tool/model.
- Make a decision on the preferred architecture of the system and confirm infrastructure readiness.
Download the Create an Architecture for AI blueprint
4. Use Info-Tech’s Off-the-Shelf AI Analysis Tool to document infrastructure decisions
2-3 hours
Input: Solution conceptual design
Output: Infrastructure requirements, Infrastructure readiness assessment
Materials: Whiteboard/Flip charts, Off-the-Shelf AI Analysis Tool, “Infrastructure” tab
Participants: Infrastructure Architect, Solution Architect, Enterprise Architect, Data Architect, ML/AI Ops Engineer
- Work with Infrastructure, Data, Solution, and Enterprise Architects to define your conceptual solution architecture.
- Define integration and storage requirements.
- Document security requirements for the solution in general and the data specifically.
- Define MLOps requirements and tools required for ML/AI pipeline deployment and production monitoring.
- Use Info-Tech’s Off-the-Shelf AI Analysis Tool's Infrastructure tab to document requirements and decisions around Data and Infrastructure Architecture.
Download the Off-the-Shelf AI Analysis Tool
What questions do you need to ask vendors when choosing the solution?
Take advantage of Info-Tech’s Rapid Application Selection Framework (RASF) to guide tool selection, but ask vendors the right questions to understand implications of having AI/ML built into the tool or a model
Data | Model | Implementation and Integration | Deployment | Security and Compliance |
|
|
|
|
|
Use Info-Tech’s Off-the-Shelf AI Analysis Tool, “Vendor Questionnaire” tab to track vendor responses to these questions.
Are you measuring impact on your processes?
Make sure that you understand the impact of the new technology on the existing business and IT processes.
And make sure your business processes are ready to take advantage of the benefits and new capabilities enabled by AI/ML.
Process automation, optimization, and improvement enabled by the technology and AI/ML-powered tools allow organizations to reduce manual work, streamline existing business processes, improve customer satisfaction, and get critical insights to assist decision making.
To take full advantage of the benefits and new capabilities enabled by the technology, make sure that business and IT processes reflect these changes:
- Processes that need to be updated.
- How the outcome of the tool or a model (e.g. predictions) is incorporated into the existing business processes and the processes that will monitor the accuracy of the outcome and monitor performance of the tool or model.
- New business and IT processes that need to be defined for the tool (e.g. chatbot maintenance, analysis of the data generated by the tool, etc.).
5. Document the Impact on Business and IT Processes
2-3 hours
Input: Solution design, Existing business and IT processes
Output: Documented updates to the existing processes, Documented new business and IT processes
Materials: Whiteboard/Flip charts, Off-the-Shelf AI Analysis Tool, “Business and IT Processes” tab
Participants: Project lead, Business stakeholders, Business analyst
- Review current business processes affected by the implementation of the AI/ML- powered tool or model. Define the changes that need to be made. The changes might include simplification of the process due to automation of some of the steps. Some processes will need to be redesigned and some processes might become obsolete.
- Document high-level steps for any new processes that need to be defined around the AI/ML-powered tool. An example of such a process would be defining new IT and business processes to support a new chatbot.
- Use Info-Tech’s Off-the-Shelf AI Analysis Tool's Business and IT Processes tab, to document process changes.
Download the Off-the-Shelf AI Analysis Tool
AI-powered Tools – Considerations
PROS:
Info-Tech Insight:No need to reinvent the wheel and build the product you can buy, but be prepared to work around tool limitations, and make sure you understand the data and the model the tool is built on. | CONS:
|
Pre-built/pre-trained models – what to keep in mind when choosing
PROS:
|
CONS:
|
Info-Tech Insight:
Using Off-the-Shelf AI models enables an agile approach to system development – faster POC and validation of ideas and approaches, but the model might not be customizable for your requirements.
Metrics
Metrics and KPIs for this project will depend on the business goals and objectives that you will identify in Step 1 of the tool selection process.
Metrics might include:
- Reduction of time spent on a specific business process. If the tool is used to automate certain steps of a business process, this metric will measure how much time was saved, in minutes/hours, compared to the process time before the introduction of the tool.
- Accuracy of prediction. This metric would measure the accuracy of estimations or predictions compared to the same estimations done before the implementation of the tool. It can be measured by generating the same prediction or estimation using the AI-powered tool or using any methods used before the introduction of the tool and comparing the results.
- Accuracy of the search results. If the AI-powered tool is a search engine, compare a) how much time it would take a user to find an article or a piece of content they were searching for using new tool vs. previous techniques, b) how many steps it took the user to locate the required article in the search results, and c) the location of the correct piece of content in the search result list (at the top of the search result list or on the tenth page).
- Time spent on manual tasks and activities. This metric will measure how much time, in minutes/hours, is spent by the employees or users on manual tasks if the tool automates some of these tasks.
- Reduction of business process steps (if the steps are being automated). To derive this metric, create a map of the business process before the introduction of the AI-powered tool and after, and determine if the tool helped to simplify the process by reducing the number of process steps.
Bibliography
Adryan, Boris. “Is it all machine learning?” Badryan, Oct. 20, 2015. Accessed Feb. 2022.
“AI-Powered Data Management Platform.” Informatica, N.d. Accessed Feb 2022.
Amazon Rekognition. “Automate your image and video analysis with machine learning.” AWS. N.d. Accessed Feb 2022.
“Artificial Intelligence (AI).” IBM Cloud Education, 3 June 2020. Accessed Feb 2022.
“Artificial intelligence (AI) vs machine learning (ML).” Microsoft Azure Documentation. Accessed Feb. 2022.
“Avante Garde in the Realm of AI” SearchUnify Cognitive Platform. Accessed Feb 2022.
“Azure Cognitive Services.” Microsoft. N.d. Accessed Feb 2022.
“Becoming an AI-fueled organization. State of AI in the enterprise, 4th edition,” Deloitte, 2020. Accessed Feb. 2022.
“Coveo Predictive Search.” Coveo, N.d. Accessed Feb 2022.
”Data and AI Leadership. Executive Survey 2022. Executive Summary of Findings.” NewVantage Partners. Accessed Feb 2022.
“Einstein Discovery in Tableau.” Tableau, N.d. Accessed Feb 2022.
Korolov, Maria. “9 biggest hurdles to AI adoption.” CIO, Feb 26, 2019. Accessed Feb 2022.
Meel, Vidushi. “What Is Deep Learning? An Easy to Understand Guide.” visio.ai. Accessed Feb. 2022.
Mitchell, Tom. “Machine Learning,” McGraw Hill, 1997.
Stewart, Matthew. “The Actual Difference Between Statistics and Machine Learning.” Towards Data Science, Mar 24, 2019. Accessed Feb 2022.
“Sentiment analysis with Cognitive Services.” Microsoft Azure Documentation. Accessed February 2022.
“Three Principles for Designing ML-Powered Products.” Spotify Blog. Oct 2019, Accessed Feb 2022.
“Video Intelligence API.” Google Cloud Platform. N.d. Accessed Feb 2022