Sandi Conrad
Principal Research Director
Info-Tech Research Group

The business needs to move quickly to adopt new ways to collect and analyze data or automate actions. IoT may be the right answer, but it can be complex and create new challenges for IT teams.

Many of these solutions are implemented by vendors as point solutions, but more organizations are recognizing they need to bring the data in-house to start driving insights.

As IoT solutions become more prolific, the need to get more involved in securing and managing these solutions has become evident.

The business is often engaging directly with the vendors to better understand how they can benefit from these solutions, and IT is often brought in when the solution is ready to go live.

When IT isn’t involved early, there may be challenges around integrations, communications, and getting access to data.

Management becomes challenging as many devices are suddenly entering the environment, which need to be inventoried, added to lifecycle management practices, and secured.

Info-Tech’s approach starts with assessing the proposed solutions to:

• Ensure they will meet the business need.
• Understand data structure for integration to central data store.
• Ensure privacy and security needs can be met.
• Determine effort and technical requirements for integration into the infrastructure and appropriate onboarding into operations.

• Point solutions may be installed and configured with support outsourced to vendors, where integrations may be light or non-existent.
• Each point solution will be owned by the business, with data used for a specific purpose, and may only require infrastructure support from the internal IT department.
• Operational needs must be met to protect the business’ investment, and without involving IT early, agreements may be signed that don’t meet long-term goals of high value at reasonable prices.
• To fully realize value from multiple disparate systems, a cohesive strategy to bring together data will be required, but with that comes a need to improve technology, determine data ownership, and improve oversight with strengthened security, privacy, and communications.
• Where IoT is becoming a major source of data, taking a piecemeal approach will no longer be enough to be successful.

IoT solutions may be chosen by the business, but to be successful and meet their requirements, a partnership with IT will ensure better communications with the service provider for a less stressful implementation with governance over security needs and protection of the organization’s data, and it will ensure that continual value is enabled through effective operations.

• Solutions managed outside of IT, whether through an operational technology team or an outsourced vender, will require a comprehensive approach that encourages collaboration, common understandings of risk, and the ability to embrace change.
• Technical expertise required will be broad and deep for a multi-solution implementation. Many types of devices, with varied connections and communications methods, will need to be architected with flexibility to accommodate changing technology and scalability needs.
• Understanding the myriad options available and where it makes sense to deploy cutting-edge vs. proven technologies, as well as edge computing and digital twins.
• External consultants specializing in IoT may need to be engaged to make these complex solutions successful, and they also need to be skilled in facilitating discussions within teams to bring them to a common understanding.
• Analysis skills and a data strategy will be key to successfully correlating data from multiple sources, and AI will be key to making sense of vast amounts of data available and be able to use it for predictive work. According to the Microsoft IoT Signals report of October 2020, “79% of organizations adopt AI as part of their IoT solution, and those who do perceive IoT to be more critical to their company’s success (95% vs. 82%) and are more satisfied with IoT (96% vs. 87%).“

1. SECURITY
   • Risk identification and assessment
   • Threat modeling – ineffective because of scale
   • Dumb, cheap endpoints without users
   • Massive attack surface
   • Data/system availability
   • Physical access to devices
   • Response to anonymized individuals
2. COMPLIANCE
   • Internal
   • External
     NIST, SOC, ISO
     Profession/industry
   • Ethics
   • Regulatory
     PII, GDPR, PIPEDA
     Audit process

3. OPERATIONAL STANDARDS
   • Industry best practices
   • Open standards vs. proprietary ones
   • Standardization
   • Automation
   • Vendor management
4. TECHNICAL OPERATIONAL MODEL
   • Platforms
   • Insourcing/outsourcing
   • Acquisition
   • Asset management
   • Patching
   • Data protection
   • Source image control
   • Software development lifecycle
   • Vendor management
   • Disposition/disposal

IoT Steering Committee Charter

IoT Solution Playbook

IT Benefits

• Aggregation of processes and data may have compelling implications for increasing effectiveness of the business, but this may also increase risk. A framework will help to drive value while putting in appropriate guardrails.
• IoT use cases may be varied within many industries, and the use of many types of sensors and devices complicates management and maintenance. A common understanding of how devices will be tracked, managed, and maintained is imperative to IT securing their systems and data.
• A pilot program to evaluate effectiveness and either reject or move forward with a plan to onboard the solution as quickly as possible will ensure quick time to value and enable immediate implementation of controls to meet operational and security requirements.

Business Benefits

• Aggregation of many disparate groups of data can provide new insights into the way an organization interacts with its clients and how clients are using products and services.
• As organizations innovate and new IoT solutions are introduced to the environment, solutions need to be evaluated quickly to determine if they’re going to meet the business case and then determine what needs to be put in place for technology, process, and policy to ensure success.
• As new solutions are introduced, anyone who may be impacted through this new data-collection process will need to be informed and feel secure in the way information is analyzed and managed. This project will provide the framework to quickly assess the risks and develop a communications plan.

• Search for real problems to solve, with visible improvement possibilities
• Don’t choose technology for technology’s sake
• Keep an eye to the future
• Strategic foresight

• Avoid the “Big Bang” approach
• Test technologies in multiple conditions
• Run inexpensive pilots
• Increase flexibility
• Technology ecosystem

• Collaborate with the community
• Gain and sustain support
• Increase uptake of city technology
• Crowdsource community ideas

• Production increases and efficiency
• Reliability as data quality increases
• New product development opportunities through better understanding of how your products are used
• New product offerings with automated data collection and analysis of aggregated data

• Improved wellness programs for employees and patients through proactive health management
  • Reduction in health care/insurance costs
  • Reduction in time off for illness
• Reduction in human error
• Improved safety – fewer equipment malfunction incidents
• Sustainability – reduction in emissions

• Cost efficiency – lower energy consumption, less waste, improved product consumption
• Reliability – reduced downtime of equipment due to condition-based maintenance
• Security – decrease in malware attacks

• # supported devices
• % of projects using IoT
• % of managed systems
• % of increase in equipment optimization

DIY Toolkit

Guided Implementation

Workshop

Consulting

Diagnostics and consistent frameworks used throughout all four options

Call #2: Define process for meeting and assessing requests.

Call #4: Define the role of the BRM & assessment criteria.

Call #6: Define assessment requirements for other IT groups.

Steering Committee

1.1 Define the committee’s roles and responsibilities in the IoT Steering Committee Charter

1.2 Define the IoT steering committee’s vision statement and mandates

1.3 Define procedures for reviewing proposals and roles and responsibilities

Intake Process

2.1 Define requirements for requesting new IoT solutions

2.2 Define procedures for reviewing proposals and projects – BA/BRM

2.3 Define procedures for reviewing proposals and projects – Data specialists

2.4 Define procedures for reviewing proposals and projects – Privacy & Security

2.5 Define procedures for reviewing proposals and projects – Infrastructure & Operations

2.6 Define service objectives and evaluation process

Proof of Value

3.1 Determine the criteria for running a proof of value

3.2 Define the template and process for running a proof of value

Create an IoT steering committee to ensure value will be realized and operational needs will be met

The goals of the steering committee should be:

• To align IoT initiatives with organizational goals. 
• To effectively evaluate, approve, and prioritize IoT initiatives.
• To approve IoT strategy & evaluation criteria.
• To reinforce and define risk evaluation criteria as they relate to IoT technology.
• To review pilot results and confirm the value achievement of approved IoT initiatives.
• To ensure the investment in IoT technology can be integrated and managed using defined parameters.

As IoT solutions move beyond their primary goals, it will be critical to evaluate the continually increasing data to mitigate risks of unintended consequences as new data sets converge. The security team will need to evaluate solutions and enforce standards.

CDO and analysts will assess opportunities for data convergence to create new insights into how your services are used.

Operational plans will include asset management, monitoring, and support to meet functional goals and manage throughout the asset lifecycle.

Each solution added to the environment will need to be chosen and architected to meet primary functions and secondary data collection.

• The committee should include team members experienced enough to provide an effective assessment of IoT projects, and to provide input and oversight regarding business value, privacy, security, operational support, infrastructure, and architectural support.
• A data specialist will be critical for evaluating opportunities to expand use of data and ensure data can be effectively validated and aggregated. Additional oversight will be needed to review aggregated data to protect against the unintended consequences of having data combined and creating personas that will identify individuals.
• Additional experts may be invited to committee meetings as appropriate, and ideas should be discussed and clarified with the business unit bringing the ideas forward or that may be impacted by solutions.
• Invite appropriate IT and business leaders to the initial meeting to gain agreement and form the governance model.

• Define the IoT vision in alignment with the organizational strategy and mission.
• Define strategy, policies and communication requirements for IoT projects.
• Assess and bring forward proposals to utilize IoT to further organizational strategy.

• Define criteria for evaluating and prioritizing proposals and projects.
• Validate the IoT proposals to ensure value drivers are understood and achievable.
• Identify opportunities to combine data sets for secondary analysis and insights.

• Evaluate data and combined data sets to avoid unintended consequences.
• Ensure security standards are adhered to when integrating new solutions.
• Reinforce privacy regulations, policy, and communications requirements.

• Identify and validate investment and resource requirements.
• Evaluate technical requirements and capabilities.
• Align IoT management requirements to operations goals within IT.

• Assess validity of pilot project plan, including success criteria.
• Identify corner cases to assess functionality and potential risks beyond core features.
• Monitor progress, evaluate results, and ensure organizational needs will be met.
• Evaluate pilot to determine if it will be moved into full production, reworked, or rejected.

Define the vision statement for the IoT committee to clarify mandate and communicate to stakeholders

A vision statement should be concise and should be in support of the overall IT strategy and organizational mission. The vision statement will be used as a high-level guide for defining and assessing proposed solutions and evaluating potential outcomes. It can be used as a limiter to quickly weed out ideas that don’t fit within the mandate, but it can also inspire new ideas.

• Support innovation
• Enable the business
• Enable operations for continual value

In order to accomplish this overall mission, they’ve created a specific IT vision statement: “Improve digital infrastructure to meet the needs of the 21st century.”

This may seem broad, and it includes not just IoT, but also the need to upgrade infrastructure to be able to enable IoT as a tool to meet the needs to collect data, take action, and better understand how people move and live within the city. You can read more of their strategy at this
link: http://onenyc.cityofnewyork.us/about/

Customer

• Executive leadership
• Business leaders

Outputs

• Risk assessment
• Approvals to proceed
• Pilot plan
• Assessment to approve for production or reject

Process

• Review proposals
• Ask questions and discuss with proposer & committee
• Review pilot & testing plan
• Engage with IT Team to define requirements

Inputs

• Request form including:
• New idea
• Business value defined
• Data collected
• Initial risk assessment
• Implementation plan
• Definition of success

Suppliers

• IT operations team
• Device and software vendors
• IT leaders
• Risk committee

The committee needs to be seen as an enabler to the business, not as a gatekeeper, so it must be thorough but responsive.

• The vision to ensure clarity of purpose.
• IoT mandates to focus the committee on assessment criteria.
• Roles, responsibilities, and assignments to engage the right people who will provide the kind of guidance needed to ensure success.
• Procedures to make the best use of each committee member’s time.
• Process flow to guide evaluations to avoid unnecessary delays while reducing organizational risks.

Steering Committee

1.1 Define the committee’s roles and responsibilities in the IoT Steering Committee Charter

1.2 Define the IoT steering committee’s vision statement and mandates

1.3 Define procedures for reviewing proposals and roles and responsibilities

Intake Process

2.1 Define requirements for requesting new IoT solutions

2.2 Define procedures for reviewing proposals and projects – BA/BRM

2.3 Define procedures for reviewing proposals and projects – Data specialists

2.4 Define procedures for reviewing proposals and projects – Privacy & Security

2.5 Define procedures for reviewing proposals and projects – Infrastructure & Operations

2.6 Define service objectives and evaluation process

Proof of Value

3.1 Determine the criteria for running a proof of value

3.2 Define the template and process for running a proof of value

• Review intake forms from the PMO or build your own from the IoT Solution Playbook:
• Start by asking for a clear picture of the solution. Ensure the requester can clearly articulate the business benefit to the solution, including what issues are being resolved and what success looks like.
• Requesters may not be expected to seek out all relevant information to make the decision.
  • Consider providing a business analyst (BA) to assist with data gathering for further assessment and to launch the review process.
  • Review may require additional steps if it is not clear the proposed solution will perform as expected and could include conversations with the vendor or a determination that a full requirements-gathering process may need to be done.
• Typically, a BA will launch the review process to have appropriate experts assess the feasibility of the solution; assess regulatory, privacy, and security concerns; and determine the level of involvement needed by IT and the project managers.
• Have options for different starting points. Some requesters may be further along in their research as they know exactly what they want, while others will be early in the idea stage. Don’t discourage innovation by creating more work than they’re able to execute.

Business goals and benefits are important to ensure the completed solution meets the intended purpose and enables appropriate collection, analysis, and use of data in the larger business context.

Ongoing operational support and service need to be considered to ensure ongoing value, and adherence to security and privacy policies is critical.

• Project sponsorship is key to moving the project ahead. Ensure the project sponsor and business owner will be in alignment on the solution and business needs.
• Note any information that will help to prioritize this project among all other requests. This will feed into implementation timing and the project management needs, resourcing, and vendor engagement required.
• Determine if a proof of value would be an asset. A proof of value can be time consuming, but it can mitigate the risks of large-scale failures.
• Ask about data collection and data type, which will be a major part of the assessment for the data team and for security, privacy, infrastructure, and operational assessments.
• Determine if any actions will need to be taken, which might include data transfer, notifications and alerts, or others. This may require additional discussions on actuators, RPA, data stores, and integrations.
• Determine if any automation will be part of the solution, as this will help to inform future discussions on power, connectivity, security, and privacy.

Download the blueprint Embed Business Relationship Management in IT if you need help to support the business in a more strategic manner.

Info-Tech Insight

Understanding the business issue more deeply can help the business analyst determine if the solution needs a review of business process as well as helping to build out the requirements well enough to improve chances of success.

The BA should be able to determine initial workload and involvement of project managers and evaluators.

• Financial – to increase profitability or reduce costs through predictive maintenance and efficiency.
• Business Development – innovation for new products, services, and methodologies
• Improve specific outcomes – typically these will be industry specific, such as improved patient health care, reduced traffic congestion or use of city resources, improved billing, or fire prevention for utility companies.

As you start to look at the bigger picture of how these different systems can bring together disparate data sets, the benefits will be harder to define, and the costs to implement this next level of data analysis can be daunting and expensive.

This doesn’t necessitate a complete alignment of data collection purposes; there may be benefits to improving operations in secondary areas such as updating HVAC systems to reduce energy costs in a hospital, though the updated systems may also include sensors to monitor air quality and further improve patient outcomes.

In these cases, there may be future opportunities to use this data in unexpected ways, but even where there aren’t, applying the same standards for security, privacy, and operations should apply.

If your organization isn’t poised to manage and make the best use of the data, see Info-Tech’s related blueprints:

• Establish Data Governance
• Create and Manage Enterprise Data Models
• Establish an Analytics Operating Model
• Build Your Data Quality Program

• Data Quality Scorecard

• Regulatory compliance.
• Data quality and validation.
• Data normalization.
• Data aggregation and analysis.

• Infrared sensors may include intelligence to count people or objects.
• Cameras might require manual counts but may provide better images.
• Good quality images may require technology to distort faces for privacy.

Self-contained or closed solutions may be quick to install and configure and may require minimal technical support from within your own IT team, but they will not provide visibility to the inner workings of the solution. This may create issues around integration and interoperability which could limit the functionality and usability beyond the point solution.

If the solution chosen is a closed system, determine how you will need to interact with the vendor to gain access to the data. Interoperability may not be an option, so work with the vendor to set up a regular cadence for accessing the data.

Questions for the vendor could include:

1. How often can we access the data? Will the vendor push it on a regular basis? Is it on demand?
2. Or will we need to pull the data? Is there an API?
3. Will the data be normalized?
4. Will the data be transferred, or will the vendor keep a historical record?
5. Are there additional fees for archiving or for data extraction?

These virtual replicas will not be necessary for every IoT application as many solutions will be very straightforward in their application. But for those complex systems, such as smart buildings, smart cities and mechanically complex projects, digital twins can be created to run multiple simulations to aid in business continuity planning, performance assessments, R&D and more.

Due to the expense and complexity of creating a full digital twin, carefully weighing the benefits, and identifying how it will be used, can help to build the business case to invest in the technology. Without the skills in house, reliance on a vendor to create the model and test scenarios will likely be part of the overall solution.

The assessment will also include understanding what data will be transferred into the model, how often it will be updated, how it will be protected and who will need to be involved in the modeling process.

Download the blueprint: Double Your Organization’s Effectiveness With a Digital Twin. if you need more information on how to leverage digital twin technology.

The significant increase in devices and applications will require a review of security practices related to IoT to understand and mitigate risks. Even if the data collected is not considered integral to the business, such as with automated HVAC systems or an aquarium monitoring system, the devices can provide an entry point to access the network.

IoT and ICS devices are functionally diverse and may include more mature solutions that have been acquired many times over. There are a wide variety of protocols that may not be recognized by vulnerability scanners as safe to operate in your environment. Many of these solutions will be agentless and may not be picked up by scanners on the network. Without knowing these devices exist or understanding the data traffic patterns, protecting the devices, data, and systems they’re attached to becomes challenging.

Discovery and vulnerability scanners tuned specifically for IoT to look for and allow unusual protocols and traffic patterns will enable these devices to operate as designed without being shut down by vulnerability scanners protecting more traditional devices and traffic on an IT network. Orphaned devices can be found and removed. Solutions that will provide detailed asset inventories and network topologies will improve vulnerability detection.

Systems that are air gapped or completely segregated may provide a layer of protection between IoT devices and the corporate network, but this may create additional difficulties in vulnerability assessment, identifying and responding to active threats, or managing the operational side. Additionally, if there are still functional connections between these systems for traffic to flow back to central repositories, operational systems, or remote connections, there are still potential threats.

If security controls are not yet documented, see Info-Tech’s related blueprints:

• Determine Your Zero Trust Readiness
• Build an Information Security Strategy
• Identify the Components of Your Cloud Security Architecture

• IT Security Diagnostic Program

As a point solution, IoT provides a means to collect large amounts of data and, if actuators are completing tasks, act quickly. When determining the use case for IoT and best fit solutions, it’s important to think about what data needs to be collected and what actions will need to be coordinated.

As the need for more than just a few IoT solutions surfaces, the complexity and potential usefulness of data increases. This can lead to significant changes to the scope of data collection, storage, and analysis, and may lead to unintended consequences.

1. Where may there be physical access to sensors and a possibility of theft, and can the data be encrypted?
2. What type of information is captured by sensors and stored in the solution?
3. Where is personally identifiable information captured, and where is it stored? How will you meet regulatory requirements such as GDPR? Where does the data fit within existing retention policies, and how long should it be kept?
4. Will there be a need to post signage or update privacy statements in response to the information being collected?

If data classification, privacy, and security controls are not yet documented, see Info-Tech’s related blueprints:

• Build a Data Privacy Program
• Embed Privacy and Security Culture Within Your Organization.
• Mature Your Privacy Operations

Look for ways of capturing information that will meet your business requirements while mitigating risk of capturing personally identifiable information. Examples would be LiDAR to capture movement instead of video, or AI to blur faces or license plate numbers at time of image capture.

This chart identifies data collected by smartphone accelerometers which could be used to identify and profile an individual and understand their behaviors.

Mobile device accelerometer data

The supporting applications will be collecting and analyzing data for each of these solutions, with most being hosted on public clouds or privately by the vendor. Access to the applications for data collection may require APIs or other middleware to transfer data outside of their application. Data transfer may be unimportant if the data collected will stand alone and never be integrated to other systems, but it will be critical if IoT plans include retrieving, aggregating, and analyzing data from most systems. If these systems are closed, determine the process to get this information, whether it’s through scheduled exports or batch transfers.

Determine if data will be backed up by the vendor or if backups are the responsibility of your team. Work with the business owner to better understand business continuity requirements to plan appropriately for data transmission, storage, and archiving.

Network and communications will vary dramatically depending on where sensors and actuators are located. On-premises solutions may rely on Wi-Fi on your network or may require an air-gapped or segregated network. External sensors may rely on public Wi-Fi, cellular, or satellite, and this may impact reliability and serviceability. If manual data collection is required, such as collecting SD cards on trail cams, who will be responsible, and will they have the tools and data repository they need to upload data manually? Are you able to work with the vendor to estimate traffic on these networks, and how will that impact costs for cellular or satellite service?

Investigate power requirements. On-premises solutions may require additional wiring, but if using wind or solar, what is the backup? If using batteries, what is the expected lifespan? Who will be monitoring, and who will be changing the batteries?

Determine monitoring requirements. Who should be responsible for performance monitoring, outages, data transmission, and validation? Is this a vendor premium service or a process to manage in-house? If managed by the vendor, discuss required SLAs and their ability to meet them.

If your organization is dealing with technical debt and older architecture which could prevent progress, see Info-Tech’s related blueprints to build out the foundation.

• Manage Your Technical Debt
• Build Your Infrastructure Roadmap
• Modernize the Network
• Map Technical Skills for a Changing Infrastructure & Operations Organization

One of the biggest challenges organizations that have already adopted IoT face is management of these systems. Without an accurate inventory, it’s impossible to know how secure the IoT systems are. Abandoned sensors, stolen cameras, and old and unpatched firmware all contribute to security risks.

Existing asset management solutions may provide the right solution, but they are limited in many cases by the discovery tools in place. Many discovery tools are designed to scan the network and may not have access to segregated or air-gapped networks or a means to access anything in the cloud or requiring remote access. Evaluate the effectiveness of current tools, and if they prove to be inadequate, look for solutions that are geared specifically to IoT as they may provide additional useful management capabilities.

IoT management tools will provide more than just inventory. They can discover IoT devices in a variety of environments, possibly adding micro-agents to access device attributes such as name, type, and date of build, and allowing metadata and tags to be added. Additionally, these solutions will provide the means to deploy firmware updates, change configuration settings, send notifications if devices are taken offline, and run vulnerability assessments. Some may even have diagnostics tools for troubleshooting and remediation.

If operational processes aren’t in place, see Info-Tech’s related blueprints to build out the foundation.

• Implement Hardware Asset Management
• Stabilize Release and Deployment Management
• Standardize the Service Desk

• IT Management & Governance Diagnostic

Regular maintenance for your team may include battery swaps, troubleshooting camera outages or intermittent sensors, or deploying patches. Understand the support requirements for the product lifecycle and who will be responsible for that work. If the vendor will be applying patches and upgrading firmware, get clarity on how often and how they’ll be deployed and validated. Ask the vendor about support documentation and offerings.

Determine the best ways of collecting inventory on the solution. Determine what the solution offers to help with this process; however, if the project plan requires specific location details to add sensors, the project list may be the best way to initially onboard the sensors into inventory.

Determine if warranty offerings are an appropriate solution for devices in each project, to schedule and record appropriate maintenance details and plan replacements as sensors reach end of life. Document dependencies for future planning.

Steering Committee

1.1 Define the committee’s roles and responsibilities in the IoT Steering Committee Charter

1.2 Define the IoT steering committee’s vision statement and mandates

1.3 Define procedures for reviewing proposals and roles and responsibilities

Intake Process

2.1 Define requirements for requesting new IoT solutions

2.2 Define procedures for reviewing proposals and projects – BA/BRM

2.3 Define procedures for reviewing proposals and projects – Data specialists

2.4 Define procedures for reviewing proposals and projects – Privacy & Security

2.5 Define procedures for reviewing proposals and projects – Infrastructure & Operations

2.6 Define service objectives and evaluation process

Proof of Value

3.1 Determine the criteria for running a proof of value

3.2 Define the template and process for running a proof of value

When should you run a proof of value?

• When it is difficult to confirm that the solution is fit for purpose.
• When the value of the solution is indeterminate.
• When the solution is early in its lifecycle and not widely proven in the marketplace.
• When scalability is questionable or unproven.
• When the solution requires customization or configuration.

Info-Tech Insight
Where a solution is well known in the market, requires minimal customization, and is proven to be fit for purpose, a shorter evaluation or conversations with reference clients or partners may be all that is necessary.

• Go back to the original request: What are the goals for implementing this solution? Has this been clearly defined with criteria for success?
• Define the technical team that will configure the solution, including vendors and technicians. Ensure the vendor fully understands your use cases and goals. Identify the level of support you’ll need to be implement and assess the solution.
• Define the testing team, including technical and business users. Complete a journey map if needed to define the use case(s) at the right level of detail.
• Ensure the test use case(s) have been defined and they all agree on the definition of success.
• Make sure the team is available to do the testing and provide feedback, as high adoption will improve feedback which will be critical to successfully implementing the full solution.
• Determine how to evaluate scalability with process, resources, and capacity.
• Evaluate the risks and obstacles to reject the solution or mitigate and prevent scope creep.
• Evaluate the vendor’s roadmap, training materials, and technical support options.

Info-Tech Insight

Additional information on building out a process for testing new technology can be found in the blueprint: Exploit Disruptive Infrastructure Technology.

“Although scope creep is not the only nemesis a project can have, it does tend to have the farthest reach. Without a properly defined project and/or allowing numerous changes along the way, a project can easily go over budget, miss the deadline, and wreak havoc on project success.” (University Alliance, Villanova University)

• Every technology is designed to solve a problem faced by somebody somewhere. For each technology that your team has decided to move forward with, identify and clearly state the problem it would solve.
• A clear problem statement is a crucial part of a new technology’s business case. It is impossible to earn buy-in from the rest of the organization without demonstrating the necessity of a solution.
• Perfection is impossible to achieve, especially during a proof of value (POV). However, knowing the pain points of the way things are done without this technology, and noting a reduction in pain and increase in efficiency and accuracy of data gathering will help in the initial feedback of the tests. Ensure the proof of value includes data validation to test accuracy.

Info-Tech Insight

Know your metrics going into the proof of value. Document performance, quality, and time to do the work and compare to metrics in the proof of value. Agree on what success looks like, to ensure that improvements are substantial enough to justify the expense and effort of implementing the solution.

• What are the project’s goals?
• What is the desired future state?
• What problems must be solved to call the POV a viable solution?
• Where will the project be rolled out? Are there any concerns about communications and power that may need to be addressed?
• Are there any risks to watch for?

Info-Tech Insight

Be sure to avoid scope creep! Remember: the goal of the proof of value project is to produce a minimum case for viability in a carefully defined area. Reserve a detailed accounting of costs and benefits for after the proof of value stage.

• The use case will help to define the scope of the project, define adjacent use cases or tasks that will be out of scope, and to contain the test to a reasonable effort and time frame, while still testing core functionality.
• Map processes based on expectations of how the solution should work, and compare these to the way things are done today. Identify if there are obvious improvements to the existing processes that if done, would change the existing results significantly. Take this into account when reviewing results. (This will also be useful if the project isn’t approved or is delayed.)
• Identify where tasks and data collection will be automated and where they will need to stay manual or require additional integrations or solutions such as RPA. These other solutions may not factor into the proof of value but will need to be identified on the solution roadmap if it goes ahead.

• Discuss steps to completion
• Effort to collect data
• Effort to validate and correct data
• Effort and ability to use the data for decision making, understanding your customers, and process improvements
• Quality of data available with current methods compared to quality and volume of data using an IoT solution

• Stakeholders: Anyone who is impacted by the new technology and who will end up using, approving, or implementing it. Identify team members who will be willing and able to test the systems for data quality, collection, and workflow improvements.
• Data analysts: Include someone who can validate the usefulness of data to meet the needs of the organization.
• Security & Privacy: Include these team members to validate their expectations of how privacy and security needs can be met.
• Infrastructure & Operations: These team members can test integrations, data collections, traffic flow, etc.
• Vendor: Discuss what part the vendor can play in setting up the solution for running the proof of value.
• Other business units: Identify business units that could benefit or be impacted by this solution. Invite them to participate in the roof of value, but remember to contain scope.

• Processes are designed to transform inputs into outputs. All business activities can be mapped into processes.
• A process map illustrates the sequence of actions and decisions that transform an input into an output.
• Effective mapping gives managers an “aerial” view of the company’s processes, making it easier to identify inefficiencies, reduce waste, and ultimately streamline operations.
• To identify business processes, have group members familiar with the affected business units identify how jobs are typically accomplished within those units.
• Ensure they have the time to test the solution and provide valid feedback.

Estimate

Estimate

Estimate

Determine

John Burwash
Senior Director, Executive Services
Info-Tech Research Group

INFO~TECH RESEARCH GROUP

External contributors
4 external contributors have asked to remain anonymous.

Jennifer Jones
Senior Research Advisor, Industry
Info-Tech Research Group

Aaron Shum
Vice President, Security, Privacy & Risk
Info-Tech Research Group

Rajesh Parab
Research Director, Applications, Data & Analytics
Info-Tech Research Group

Frank Sargent
Senior Director Practice Lead, Security, Privacy & Risk
Info-Tech Research Group

Scott Young
Principal Research Advisor, Infrastructure
Info-Tech Research Group

Rocco Rao
Director, Research Advisor, Industry
Info-Tech Research Group

A concise guide to understanding how IoT applications will create value in your firm.