Get Instant Access
to This Blueprint

Applications icon

Define a Release Management Process to Deliver Lasting Value

Use your releases to drive business value and enhance the benefits delivered by your move to Agile.

  • Your software platforms are a key enabler of your brand. When there are issues releasing, this brand suffers. Client confidence and satisfaction erode.
  • Your organization has invested significant capital in creating a culture product ownership, Agile, and DevOps. Yet the benefits from these investments are not yet fully realized.
  • Customers have more choices than ever when it comes to products and services. They require features and capabilities delivered quickly, consistently, and of sufficient quality otherwise they will look elsewhere.

Our Advice

Critical Insight

  • Eliminate the need for dedicating time for off-hour or weekend release activities. Use a release management framework for optimizing release-related tasks, making them predictable and of high quality.

Impact and Result

  • Develop a release management framework that efficiently and effectively orchestrates the different functions supporting a software’s release.
  • Use the release management framework and turn release-related activities into non-events.
  • Use principles of continuous delivery for converting your release processes from an overarching concern to a feature of a high-performing software practice.

Define a Release Management Process to Deliver Lasting Value Research & Tools

1. Define a Release Management Process to Deliver Lasting Value Deck – A step-by-step document that walks you through how to develop and implement a release management framework that takes advantage of continuous delivery.

This presentation documents the Info-Tech approach to defining your application release management framework.

It includes the guidance from our research along with exercises that are specifically defined to help the user build a release management framework.

2. Define a Release Management Process to Deliver Lasting Value Template – Use this template to help you define, detail, and make a reality your strategy in support of your application release management framework.

The template gives the user a guide to the development of their application release management framework.

3. Define a Release Management Process to Deliver Lasting Value Workbook – This workbook documents the results of the exercises contained in the blueprint and offers the user a guide to development of their release management framework.

This workbook is designed to capture the results of your exercises from the Define a Release Management Process to Deliver Lasting Value blueprint.


Member Testimonials

After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve. See our top member experiences for this blueprint and what our clients have to say.

10.0/10


Overall Impact

$12,599


Average $ Saved

10


Average Days Saved

Client

Experience

Impact

$ Saved

Days Saved

Packaging Machinery Manufacturers Institute

Guided Implementation

10/10

$12,599

10


Workshop: Define a Release Management Process to Deliver Lasting Value

Workshops offer an easy way to accelerate your project. If you are unable to do the project yourself, and a Guided Implementation isn't enough, we offer low-cost delivery of our project workshops. We take you through every phase of your project and ensure that you have a roadmap in place to complete your project successfully.

Module 1: Define the Current Situation

The Purpose

  • Document the existing release management process and current pain points and use this to define the future-state framework.

Key Benefits Achieved

  • Gain an understanding of the current process to confirm potential areas of opportunity.
  • Understand current pain points so that we can build resolution into the new process.

Activities

Outputs

1.1

Identify current pain points with your release management process. If appropriate, rank them in order of most to least disruptive.

  • Understanding of pain points, their root causes, and ranking.
1.2

Use the statement of quality and current pain points (in addition to other considerations) and outline the guiding principles for your application release management framework.

  • Built guiding principles for application release management framework.
1.3

Brainstorm a set of metrics that will be used to assess the success of your aspired-to application release management framework.

  • Created set of metrics to measure the effectiveness of the application release management framework.

Module 2: Define Standard Release Criteria

The Purpose

  • Build sample release criteria, release contents, and standards for how it will be integrated in production.

Key Benefits Achieved

  • Define a map to what success will look like once a new process is defined.
  • Develop standards that the new process must meet to ensure benefits are realized.

Activities

Outputs

2.1

Using an example of a product known to the team, list its criteria for release.

  • Completed Workbook example highlighting releasability.
2.2

Using an example of a product known to the team, develop a list of features and tasks that are directly and indirectly important for either a real or hypothetical upcoming release.

  • Completed Workbook example defining and detailing feature and task selection.
2.3

Using an example of product known to the team, map out the process for its integration into the release-approved code in production. For each step in the process, think about how it satisfies guiding principles, releasability and principles of continuous anything.

  • Completed Workbook example defining and detailing the integration step.

Module 3: Define Acceptance and Deployment Standards

The Purpose

  • Define criteria for the critical acceptance and deployment phases of the release.

Key Benefits Achieved

  • Ensure that releases will meet or exceed expectations and meet user quality standards.
  • Ensure release standards for no / low risk deployments are recognized and implemented.

Activities

Outputs

3.1

Using an example of product known to the team, map out the process for its acceptance. For each step in the process, think about how it satisfies guiding principles, releasability and principles of continuous anything.

  • Completed Workbook example defining and detailing the acceptance step.
3.2

Using an example of product known to the team, map out the process for its deployment. For each step in the process, think about how it satisfies guiding principles, releasability and principles of continuous anything.

  • Completed Workbook example defining and detailing the deployment step.

Module 4: Implement the Strategy

The Purpose

  • Define your future application release management process and the plan to make the required changes to implement.

Key Benefits Achieved

  • Build a repeatable process that meets the standards defined in phases 2 and 3.
  • Ensure the pain points defined in Phase 1 are resolved.
  • Show how the new process will be implemented.

Activities

Outputs

4.1

Develop a plan and roadmap to enhance the integration, acceptance, and deployment processes.

  • List of initiatives to reach the target state
  • Application release management implementation roadmap

Define a Release Management Process for Your Applications to Deliver Lasting Value

Use your releases to drive business value and enhance the benefits delivered by your move to Agile.

Analyst Perspective

Improving your release management strategy and practices is a key step to fully unlock the value of your portfolio.

As firms invest in modern delivery practices based around product ownership, Agile, and DevOps, organizations assume that’s all that is necessary to consistently deliver value. As organizations continue to release, they continue to see challenges delivering applications of sufficient and consistent quality.

Delivering value doesn’t only require good vision, requirements, and technology. It requires a consistent and reliable approach to releasing and delivering products and services to your customer. Reaching this goal requires the definition of standards and criteria to govern release readiness, testing, and deployment.

This will ensure that when you deploy a release it meets the high standards expected by your clients and delivers the value you have intended.

Dr. Suneel Ghei

Principal Research Director, Application Development

Info-Tech Research Group

Executive Summary

Your Challenge

  • Your software platforms are a key enabler of your brand. When there are issues releasing, the brand suffers. Client confidence and satisfaction erode.
  • Your organization has invested significant capital in creating a culture of product ownership, Agile, and DevOps. Yet the benefits from these investments are not yet fully realized.
  • Customers have more choices than ever when it comes to products and services. They require features and capabilities delivered quickly, consistently, and of sufficient quality, otherwise they will look elsewhere.

Common Obstacles

  • Development teams are moving faster but then face delays waiting for testing and deployment due to a lack of defined release cycle and process.
  • Individual stages in your software development life cycle (SDLC), such as code collaboration, testing, and deployment, have become leaner, but the overall complexity has increased since many products and services are composed of many applications, platforms, and processes.
  • The specifics of releasing products is (wrongly) classified as a technical concern and not a business concern, hindering the ability to prioritize improved release practices.

Info-Tech's Approach

  • Develop a release management framework that efficiently and effectively orchestrates the different functions supporting a software’s release.
  • Use the release management framework and turn release-related activities into non-events.
  • Use principles of continuous delivery for converting your release processes from an overarching concern to a feature of a high-performing software practice.

Executive Summary

Info-Tech Insights

Turn release-related activities into non-events.

Eliminate the need for dedicating time for off-hour or weekend release activities. Use a release management framework for optimizing release-related tasks, making them predictable and of high quality.

Release management is NOT a part of the software delivery life cycle.

The release cycle runs parallel to the software delivery life cycle but is not tightly coupled with it. The act of releasing begins at the point requirements are confirmed and ends when user satisfaction is measurable. In contrast, the software delivery life cycle is focused on activities such as building, architecting, and testing.

All releases are NOT created equal.

Barring standard guiding principles, each release may have specific nuances that need to be considered as part of release planning.

Your release management journey

  1. Optimize Applications Release Management - Set a baseline release management process and organization.
  2. Modernize Your SDLC - Move your organization to Agile and increase throughput to feed releases.
  3. Deliver on Your Digital Product Vision - Understand the practices that go into delivering products, including articulating your release plans.
  4. Automate Testing to Get More Done - Create the ability to do more testing quickly and ensure test coverage.
  5. Implement DevOps Practices That Work - Build in tools and techniques necessary for release deployment automation.
  6. Define a Release Management Process to Deliver Lasting Value (We Are Here)

Define a Release Management Process for Your Applications to Deliver Lasting Value

Use your releases to drive business value and enhance the benefits delivered by your move to Agile.

Executive Brief

Your software delivery teams are expected to deliver value to stakeholders in a timely manner and with high quality

Software delivery teams must enable the organization to react to market needs and competitive changes to improve the business’ bottom line. Otherwise, the business will question the team’s competencies.

The business is constantly looking for innovative ways to do their jobs better and they need support from your technical teams.

The increased stress from the business is widening the inefficiencies that already exist in application release management, risking poor product quality and delayed releases.

Being detached from the release process, business stakeholders do not fully understand the complexities and challenges of completing a release, which complicates the team’s communication with them when issues occur.

IT Stakeholders Are Also Not Satisfied With Their Own Throughput

  • Only 29% of IT employees find application development throughput highly effective.
  • Only 9% of organizations were classified as having highly effective application development throughput.
  • Application development throughput ranked 37th out of 45 core IT processes in terms of effectiveness.

(Info-Tech’s Management and Governance Diagnostic, N=3,930)

Your teams, however, struggle with core release issues, resulting in delayed delivery (and disappointed stakeholders)

Implementing tools on top of an inefficient pipeline can significantly magnify the existing release issues. This can lead to missed deadlines, poor product quality, and business distrust with software delivery teams.

COMMON RELEASE ISSUES

  1. Local Thinking: Release decisions and changes are made and approved without consideration of the holistic system, process, and organization.
  2. No Release Cadence: Lack of process governance and oversight generates unpredictable bottlenecks and load and ill-prepared downstream teams.
  3. Mismanagement of Releases: Program management does not accommodate the various integrated releases completed by multiple delivery teams.
  4. Poor Scope Management: Teams are struggling to effectively accommodate changes during the project.

The bottom line: The business’ ability to operate is dictated by the software delivery team’s ability to successfully complete releases. If the team performs poorly, then the business will do poorly as well. Application release management is critical to ensure business expectations are within the team’s constraints.

As software becomes more embedded in the business, firms are discovering that the velocity of business change is now limited by how quickly they can deploy.” – Five Ways To Streamline Release Management, J.S. Hammond

Historically, managing releases has been difficult and complicated…

Typically, application release management has been hard to coordinate because…

  • Software has multiple dependencies and coordinating their inclusion into a deployable whole was not planned.
  • Teams many be spending too much time on features that are not needed any longer.
  • Software development functions (such as application architecture, test-first or test-driven design, source code integration, and functional testing) are not optimized.
  • There are no agreed upon service-level contracts (e.g. expected details in requirements, adequate testing, source control strategy) between development functions.
  • The different development functions are not integrated in a holistic style.
  • The different deployment environments have variability in their configuration, reducing the reliability of testing done in different environments.
  • Minimum thresholds for acceptable quality of development functions are either too low (leading to adverse outcomes down stream) or too high (leading to unnecessary delays).

…but research shows being effective at application release management increases your throughput

Research conducted on Info-Tech's members shows overwhelming evidence that application throughput is strongly tied to an effective application release management approach.

The image shows a scatter plot, with Release Management Effectiveness on the x-axis and Application Development Throughput Effectiveness on the Y-axis. The graph shows a steady increase.

(Info-Tech Management & Governance Diagnostic, since 2019; N=684 organizations)

An application release management framework is critical for effective and timely delivery of software

A well-developed application release management framework is transformative and changes...

From To
Short-lived projects Ongoing enhancements supporting a product strategy
Aiming for mandated targets Flexible roadmaps
Manual execution of release processes Automating a release pipeline as much as possible and reasonable
Manual quality assurance Automated assessment of quality
Centralized decision making Small, independent release teams, orchestrated through an optimized value stream

Info-Tech Insight: Your application release management framework should turn a system release into a non-event. This is only possible through the development of a holistic, low-risk and standardized approach to releasing software, irrespective of their size or complexity.

Robust continuous “anything” requires proficiency in five core practices

A continuous anything evaluation should not be a “one-and-done” event. As part of ongoing improvements, keep evolving it to make it a fundamental component of a strong operational strategy.

Continuous Anything

  • Automate where appropriate
    • Automation is not a silver bullet. All processes are not created equal; and therefore, some are not worthy of being automated.
  • Control system variables
    • Deploying and testing in environments that are apple to apple in comparison reduces the risk of unintended outcomes from production release.
  • Measure process outcomes
    • A process not open to being measured is a process bound to fail. If it can be measured, it should be, and insights found should be used for improving the system.
  • Select smaller features batches
    • Smaller release packages reduce the chances of cognitive load associated with finding root causes for defects and issues that may result as post-production incidents.
  • Reduction of cycle time
    • Identification of waste in each stage of the continuous anything process helps in lowering cost of operations and results in quicker generation of value for stakeholders.

Invest time in developing an application release management framework for your development team(s) with a continuous anything mindset

An application release management framework converts a set of features and make them ready for releasability in a low-risk, standardized, and high-quality process.

The image shows a diagram titled Application Release Engineering From Idea to Product, which illustrates the process.

A continuous anything (integration, delivery, and deployment) mindset is based on a growth and improvement philosophy, where every event is considered a valid data point for investigation of process efficiency.

Diagram adapted from Continuous Delivery in the Wild, Pete Hodgson, Published by O'Reilly Media, Inc., 2020

Related Info-Tech Research

Streamline Application Maintenance

  • Justify the necessity of streamlined maintenance. Gain a grounded understanding of stakeholder objectives and concerns and validate their achievability against the current state of the people, process, and technologies involved in application maintenance.
  • Strengthen triaging and prioritization practices. Obtain a holistic picture of the business and technical impacts, risks, and urgencies of each accepted maintenance request to justify its prioritization and relevance within your backlog. Identify opportunities to bundle requests together or integrate them within project commitments to ensure completion.
  • Establish and govern a repeatable process. Develop a maintenance process with well-defined stage gates, quality controls, and roles and responsibilities, and instill development best practices to improve the success of delivery.

“Releasability” (or release criteria) of a system depends upon the inclusion of necessary building blocks and proof that they were worked on

There is no standard definition of a system’s releasability. However, there are common themes around completions or assessments that should be investigated as part of a release:

  • The range of performance, technical, or compliance standards that need to be assessed.
  • The full range of test types required for business approval: unit tests, acceptance tests, security test, data migration tests, etc.
  • The volume-criticality mix of defects the organization is willing to accept as a risk.
  • The best source and version control strategy for the development team. This is mostly a function of the team's skill with using release branches and coordinating their work artifacts.
  • The addition of monitoring points and measures required for evaluations and impact analysis.
  • The documentation required for audit and compliance.
  • External and internal dependencies and integrations.
  • Validations, approvals, and sign-offs required as part of the business’ operating procedure.
  • Processes that are currently carried out outside and should be moved into the pipeline.
  • Manual processes that may be automated.
  • Any waste activities that do not directly contribute to releasability that can be eliminated from the development process.
  • Knowledge the team has regarding challenges and successes with similar software releases in the past.

Releasability of a system is different than governing principles for application release management

Governing principles are fundamental ways of doing something, which in this case is application release management, while releasability will generally have governing principles in addition to specific needs for a successful release.

Example of Governing Principles

  • Approval from Senior Director is necessary before releasing to production
  • Production deployments can only be done in off-hours
  • We will try to automate processes whenever it is possible for us to do so
  • We will use a collaborative set of metrics to measure our processes

Examples of Releasability Criteria

  • For the upcoming release, add performance testing for Finance and Budget Teams’ APIs
  • Audit and compliance documentation is required for this release
  • Automation of manual deployment
  • Use trunk-based source code management instead of feature-based

Regulated industries are not more stable despite being less nimble

A pervasive myth in industry revolves around the misperception that continuous anything and nimble and non-event application release management is not possible in large bureaucratic and regulated organizations because they are risk-averse.

"We found that external approvals were negatively correlated with lead-time, deployment frequency and restore time, and had no correlation with change failure rate. In short, approval by an external body (such as a manager or Change Approval Board) simply doesn’t work to increase the stability of production systems…However, it certainly slows things down. It is in fact worse than having no change approval process at all." – Accelerate by Gene Kim, Jez Humble, and Nicole Forsgren

Many organizations reduce risk in their product release by adopting a paternalistic stance by:

  • Requiring manual sign-offs from senior personnel who are external to the organization.
  • Increasing the number and level of authorization gates.
  • Staying away from change and preferring to stick with what has worked in the past.

Despite the prevalence of these types of responses to risk, the evidence is that they do not work and are in fact counter-productive because they:

  • Create blocks to frequent releases.
  • Introduce procedural complexity to each release and in effect make them “bigger.”
  • Prefer process over people (and trusting them). Increase non-value-add scrutiny and reporting.

There is a persistent misunderstanding about continuous anything being only an IT engineering practice

01

At the enterprise level, continuous anything focuses on:

  • Visibility of final value being provided in a high-quality and expedited manner
  • Ensuring efficiency in the organization’s delivery framework
  • Ensuring adherence to established governance and risk mitigation strategy

02

Focus of this blueprint

At the product level, continuous anything focuses on:

  • Reliability of the product delivery system
  • Use of scientific evidence for continuous improvement of the product’s delivery system
  • Orchestration of different artifacts into a single whole

03

At the functional level, continuous anything focuses on*:

  • Local functional optimization (functions = software engineering, testing, application design)
  • Automation of local functions
  • Use of patterns for standardizing inputs and functional areas

*Where necessary, practices at this level have been mentioned.

Related Info-Tech Research

Implement DevOps Practices That Work

  • Be DevOps, rather than do DevOps. DevOps is a philosophy, not an industry framework. Your organization’s culture must shift toward system-wide thinking, cross-function collaboration, and empathy.
  • Culture, learning, automation, integrated teams, and metrics and governance (CLAIM) are all critical components of effective DevOps.

Automate Testing to Get More Done

  • Optimize and automate SDLC stages to recover team capacity. Recognize that automation without optimization is a recipe for long-term pain. Do it right the first time.
  • Optimization and automation are not one-hit wonders. Technical debt is a part of software systems and never goes away. The only remedy is constant vigilance and enhancements to the processes.

The seeds of a good release are sown even before work on it begins

Pre-release practices such as requirements intake and product backlog management are important because:

  • A standard process for documentation of features and requirements helps reduce “cognitive dissonance” between business and technology teams. Clearly articulated and well-understood business needs are fundamental ingredients of a high-quality product.
  • Product backlog management done right ensures the prioritized delivery of value to stakeholders. Features can become stale or get a bump in importance, depending upon evolving circumstances. Prioritizing the backlog is, therefore, critical for ensuring time, effort, and budget are spent on things that matter.

Related Info-Tech Research

Deliver on Your Digital Product Vision

A vision without tactics is an unsubstantiated dream, while tactics without a vision is working without a purpose. You need to have a handle on both to achieve outcomes that are aligned with the needs of your organization.

Improve Requirements Gathering

  • Enhanced requirements analysis will lead to tangible reductions in cycle time, reduce project overhead, and strengthen the relationship between business and IT as more and more applications satisfy stakeholder needs.
  • More importantly, the applications delivered by IT will meet all the must-have and at least some of the nice-to-have requirements, allowing end users to successfully execute their day-to-day responsibilities.

Continuous anything has three fundamental guiding principles

Continuous anything is:

  1. A Lean approach. The components used must be continually tuned. We must focus on organizing development activities efficiently and reduce lead time of the development cycle by simplifying complex structures, eliminating overheads, and making hand-offs as seamless as possible.
  2. Not a fixed procedure or a set of tools. It cannot be bought and installed. It is an approach that, if properly followed, forces delivery teams to become aware of their deficiencies and then provides guiding principles to eliminate them.
  3. An engineering discipline based on the scientific principle of falsifiability. One failed test in any stage of the delivery cycle is more important as an indicator of a release candidate’s quality than a series of positive test cases.

Develop your application release management framework reference guide with Info-Tech’s template

Info-Tech’s Define a Release Management Process to Deliver Lasting Value Template helps you:

  • Level set your release management goals and objectives.
  • Define your foundational practices and ownership.
  • Describe the practices and tactics to automate delivery, including the role accountable for its implementation and execution and the tools involved.
  • Lay out your release management pipeline implementation roadmap and initiatives.

INFO-TECH DELIVERABLE

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Guided Implementation

What does a typical GI on this topic look like?

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

Best-Practice Toolkit Application Release Management Framework Quick Guide
Guided Implementations

Call #1: Identify current pain points with your release management process. If appropriate, rank them in order of most to least disruptive.

Call #2: Outline the guiding principles for your application release management framework and brainstorm metrics for it.

Call #3: Understand the concept of Releasability Criteria and determine approach to feature/task selection.

Call #4: Continuous Integration

Call #5: Continuous Acceptance

Call #6: Continuous Deployment

Call #7: Roadmap to the Future

Outcome

  • Understand application release planning for your organization.
  • Set up a continuous delivery mechanism for orchestrating the flow of artifacts from different sources.
  • Confirm ownership of the implementation and execution of release management framework.
  • Create list of practices to consider and implement as part of post-release system monitoring.

Workshop Overview

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

Module 1 Module 2 Module 3 Module 4
Activities Define Your Current Situation Build Standard Release Criteria Define Acceptance and Deployment Standards Implement the Strategy

1.1 Identify current pain points with your release management process. If appropriate, rank them in order of most to least disruptive.

1.2 Use the statement of quality and current pain points (in addition to other considerations), and outline the guiding principles for your application release management framework.

1.3 Brainstorm a set of metrics that will be used to assess the success of your aspired-to application release management framework.

2.1 Using an example of a product known to the team, list its criteria for release.

2.2 Using an example of a product known to the team, develop a list of features and tasks that are directly and indirectly important for either a real or a hypothetical upcoming release.

2.3 Using an example product known to the team, map out the process for its integration into the release-approved code in production. For each step in the process, think about how it satisfies guiding principles, releasability, and principles of continuous anything.

3.1 Using an example product known to the team, map out the process for its acceptance. For each step in the process, think about how it satisfies guiding principles, releasability, and principles of continuous anything.

3.2 Using an example product known to the team, map out the process for its deployment. For each step in the process, think about how it satisfies guiding principles, releasability, and principles of continuous anything.

4.1 Develop a plan and roadmap to enhance the integration, acceptance, and deployment processes.
Deliverables
  1. Understanding of pain points, their root causes and ranking
  2. Guiding principles for application release management framework
  3. Set of metrics to measure the effectiveness of the application release management framework
  1. Completed Workbook example highlighting releasability
  2. Completed Workbook example defining and detailing feature and task selection
  3. Completed Workbook example defining and detailing the integration step
  1. Completed Workbook example defining and detailing the acceptance step
  2. Completed Workbook example defining and detailing the deployment step
  1. List of initiatives to reach the target state
  2. Application release management implementation roadmap

Phase 1

Establish a sense of the status quo of your application release process.

Define Your Current Situation

1.1: Pain Points in Status Quo

1.2: Guiding Principles

1.3: Metrics

Define Standard Release Criteria

2.1: Understand Criteria for Release

2.2: Pick the Right Features and Tasks

2.3: Integrate

Define Standards for Acceptance and Deployment

3.1: Acceptance

3.2: Deployment

Implement the Strategy

4.1: Build a Plan to Implement the Strategy

This phase will walk you through the following activities:

  • Current-state assessment
  • Challenges
  • Guiding principles
  • Metrics

This phase involves the following roles:

  • Product Owner
  • Product Managers
  • Application Architect
  • Software Development Teams

1.1 List the current pain points in your application release process

2 hours

Input

  • Business objectives
  • Operational objectives
  • Regulatory responsibilities

Output

  • Ranked list of pain points in current application release management process

Materials

  • Whiteboard/Flip Charts

Participants

  • Product Owner/Product Manager
  • Software Engineering Team
  • Software Testing Team
  • Infrastructure Teams
  • Compliance and Regulatory Teams
  • Security and Data Privacy Teams
  1. Using lessons learned from previous releases, list the current obstacles that impede successful release of software.
  2. For each pain point, identify the root cause.
  3. Rank the pain points by most to least prohibitive.
    • For ranking, use a quantitative scale (1 to 10), qualitative (high, medium, low), or any other heuristic that makes sense.

Download the Define a Release Management Process to Deliver Lasting Value Workbook for documenting the outcomes of this exercise

Example: Pain points, root causes, and ranking

Pain Points Root Causes Ranking
Post-release smoke test defect rates are higher than in the past
  • Acceptance stage is manual and that deters team from running exhaustive-enough tests before declaring product ready for production.
High
Deployment environments have misconfigured variables
  • Different environments have different settings, leading to assumptions about system functioning in production.
High
Integration tests are brittle
  • Unit tests are not done all the time.
  • Automated testing as part of integration is not always done.
Medium

The importance of values and principles

  • Modern development practices (like Agile, lean, DevOps) are values and principles based. This supports the move away from command-and-control-based management to self-organizing teams.
  • Teams will make decisions that are consistent with values and principles, rather than simply following instructions.
  • These practices help an organization to unlock the wisdom of the crowd when implemented properly.

Values

Core Beliefs

Values capture the important core beliefs you want to instill in your team.

Principles

Heuristics

A method for solving a problem or deciding under pressure. We must accept they are both fallible and conducive to learning.

1.2 Establish guiding principles for the future-state application release management framework

1 hour

Input

  • Business objectives
  • Operational objectives
  • Regulatory responsibilities

Output

  • Guiding principles

Materials

  • Whiteboard/Flip Charts

Participants

  • Product Owner/Product Manager
  • Software Engineering Team
  • Software Testing Team
  • Infrastructure Teams
  • Compliance and Regulatory Teams
  • Security and Data Privacy Teams

Consider the following questions in this exercise:

  • How does your organization empower and tolerate cultural shift and other high-level change? Is lean an accepted approach to development?
  • Have other areas in your organization already implemented some lean and Agile principles?
  • Will your principles be well accepted by your organization?
  • How will they be supported and enforced?
  • Do your proposed principles consider the concerns of both development teams and business stakeholders?
  • How do these principles align with your current business priorities, strategies, and objectives?
  • Will all development functions (e.g. developers, testers, operations) agree on these guiding principles?

These principles will define how your development teams should behave, how your process should be executed, and how your tools should be configured. Keep them at the top of your mind as you define your release management future state.

Example: Guiding principles

Category Principle Rationale Impact
Feature Selection
  • Features selected for release candidate will be selected only if they have been prioritized in the backlog and have business approval for their assigned priority.
  • Spending time on non-prioritized requirements can end up in a waste of time and effort, with the added risk that the requirement might change enough to create additional waste.
  • The quality of our work is not only driven by sufficient testing but also by meeting the feature expectations of our users. Wasting time on non-value-add requirements takes away time from other required features AND does not provide value asked for.
Process Engineering
  • Every step that can be automated should be automated.
  • Manual work steps are error prone and dependent on the individual performing the task. Automation tools reduce errors and time spent waiting for others to complete a task.
  • Automated tasks will be performed more quickly and regularly and with reduced errors. Team members will spend less time doing low-value-added tasks.
System Development
  • For existing monolithic applications, enhancements will include 10% effort toward architecting them into modular components.
  • Continuous anything works best when the system is engineered in a modular fashion. Without it, the spirit of continuity, which is heavily inspired by small pieces of work, gets stymied.
  • Modularity makes a system flexible and allows teams to, without much risk, pick the tasks and work that provides most value.

Cut once, measure twice

Measure every step on the way to your true north

Introducing a change demands that it be assessed and measured, else your organization could end up spending good money after bad. Start with establishing a baseline.

The image is a graphic with text at the centre, and then notes arranged around it, sorted into 5 categories.

All 5 fingers are not equal…

…and so is the case with a holistic release management process.

Identify each function that makes up your release process and select metrics that make the most sense for each of them.

We do not selectively choose between these measures – they all together create the value you are after. After all, being fast but failing frequently or being extremely stable and very slow are not good outcomes.

Two typical release outcomes measured are quality and efficiency

Quality can be determined by measuring a release pipeline’s stability:

  • Change Failure Rate: How often do we introduce a defect in any one part of our process?
  • Failure Recovery Rate: How long does it take for us to fix a problem in production? Further analysis can be done on time taken per feature priority level or time taken for fixing a problem for a specific delivery platform (e.g. mobile versus pure web).

Efficiency can be measured by throughput, which is a measure of the efficiency with which we produce new software:

  • Frequency: How often we can release changes into production?
  • Lead Time: How long it takes to go from commit to a releasable outcome?

Ideally, these metrics are monitored throughout the process and easily automated.

Related Info-Tech Research

Select and Use SDLC Metrics Effectively

  • Your organization wants to implement (or revamp existing) software delivery metrics to monitor performance as well as achieve its goals.
  • You know that metrics can be a powerful tool for managing team behavior and mismanagement, which can lead to unintended consequences that will harm your organization.
  • You need an approach for selecting and using effective software development lifecycle (SDLC) metrics that will help your organization to achieve its goals while minimizing the risk of unintended consequences.
  • Metrics are powerful, dangerous, and often mismanaged, particularly when they are tied to reward or punishment. To use SDLC metrics effectively, know the dangers, understand good practices, and then follow Info-Tech‘s TAG (team-oriented, adaptive, and goal-focused) approach to minimize risk and maximize impact.

1.3 Brainstorm a set of metrics that can be used to evaluate the effectiveness of the future-state application release management framework

1 hour

Input

  • Business objectives
  • Operational objectives
  • Regulatory responsibilities

Output

  • Metrics

Materials

  • Whiteboard/Flip Charts

Participants

  • Product Owner/Product Manager
  • Software Engineering Team
  • Software Testing Team
  • Infrastructure Teams
  • Compliance and Regulatory Teams
  • Security and Data Privacy Teams

Metrics can be defined in three critical areas:

  • Organizational objectives focused. How does your organization measure success from a product perspective (e.g. product revenue, product operational costs, client satisfaction)?
  • Development team metrics. How do you measure success of your product releases in terms of quality and efficiency (e.g. incidents resulting from a release, delivery timeline versus original estimate, delivered scope versus estimate)?
  • Operational metrics. How do you measure success of your applications in production (e.g. availability, mean time to resolve issues)?

Based on these areas brainstorm the following:

  1. Five metrics for each area that can be used to measure the effectiveness of your release management. 30 minutes
  2. How will each measurement be calculated for each release? 15 minutes
  3. How will the metric be reported for each release? 15 minutes
  4. Who will be the owner of the reporting and for actions to improve? 15 minutes
  5. Pare the list down to three to four metrics that best cover the areas and offer the best mix of measurement, reporting, and opportunities to continuously improve. 45 minutes

Measure Success: Establish a baseline (continued)

The image shows a graphic, with notes arranged in a chart, under the column headings: Product Owner; Software Engineers; Software Testers; Infrastructure; Database Programmer. Each note has the text <measurement data-verified= in it.">

By the end of this Phase, you should...

  • Have a ranked list of pain points that are challenging your release management process currently, along with their root causes.
  • Have a list of guiding principles for the future-state application release management framework.
  • Have a list of metrics for assessing the effectiveness of the future-state application release management framework.

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com

1-888-670-8889

Phase 2

Understand the idea of releasability, picking the right features for the release candidate, and fine-tune the code integration step.

Define Your Current Situation

1.1: Pain Points in Status Quo

1.2: Guiding Principles

1.3: Metrics

Define Standard Release Criteria

2.1: Understand Criteria for Release

2.2: Pick the Right Features and Tasks

2.3: Integrate

Define Standards for Acceptance and Deployment

3.1: Acceptance

3.2: Deployment

Implement the Strategy

4.1: Build a Plan to Implement the Strategy

This phase will walk you through the following activities:

  • Understand the concept of releasability
  • Select the necessary features and tasks for ensuring release success
  • Fine-tune the code integration process

This phase involves the following roles:

  • Product Owner
  • Product Managers
  • Application Architect
  • Software Development Teams

Releasability of a system depends upon the inclusion of necessary building blocks and proof that they were worked on

Releasability refers to the set of things that need to be completed or improved upon to deliver a software. There is no standard definition of a system’s releasability. However, there are common themes that run across all of them and should be investigated as part of a release; the themes act like guiding principles for a release.

Building Blocks Examples
Business
  • 100% of high-priority requirements delivered, 60% of mediums and 2 critical defects from August 20, 2020 release
  • Business requirements are documented
Technology
  • All approvals and sign-offs are completed
  • Communication of post-release expectations is complete
  • Creating release notes for distribution and education is complete
  • Delivery standards such as Agile or DevOps are used
  • Source and version control approach has been followed
  • External integrations and their validations successfully tested
  • Technical requirements documented completely
  • Range and scope of test types determined via consensus between different teams have been successfully executed
  • Metrics for assessing efficiency, reliability, and quality of software delivery plan recorded
  • Planned automating of manual delivery processes
Governance
  • Audit trails
  • Compliance with regulations
  • Privacy of data
  • Consent management

The list of items in a theme(s) may not change from release to release but their implementation may. In some cases, it is also possible that items previously never dealt with may be added to a theme and their implementation may evolve during the release cycle (following the principles of continuous anything).

Example: Releasability criteria

Example scenario:

  • A healthcare organization is adding data integration modules to their existing data warehouse.
  • The modules will be used for real-time data reporting and will need to follow Personal Health Information Protection Act (PHIPA) rules.
  • The primary users of the reports will be physicians and nurses.
  • Due to the volume of data each report will display, it is important to ensure the system is highly performant.
  • The system also needs to be secure because it contains sensitive data.
Building Blocks Primary Owner Releasability Success Criteria
Business
  • 100% of high-priority requirements delivered, 60% of mediums and 2 critical defects from production
  • Business requirements are documented
Technology
  • Technology teams will get sign-offs from chief surgeon and head nurse pre-deployment
  • Communication of SLAs is completed
  • Creating release notes for distribution and education is complete
  • Technical requirements documented completely
  • Load, stress, and soak testing is completed and signed off
  • Metrics for assessing efficiency, reliability, and quality of software delivery plan recorded
Governance
  • Compliance with PHIPA
  • Privacy of data
  • Consent management

2.1 Releasability/release criteria as a concept

2 hours

Input

  • Guiding principles
  • Documented backlog
  • Non-functional requirements
  • Functional requirements

Output

  • List of criteria for releasability for specific release

Materials

  • Whiteboard/Flip Charts

Participants

  • Product Owner/Product Manager
  • Software Engineering Team
  • Software Testing Team
  • Infrastructure Team
  • Database Engineering Team
  1. Using an example of a product known to the team, list its release criteria.

*Release criteria are specific to a release and should not be confused with guiding principles (which have a more permanent or long-term relevance at the organizational level).

Complex products may have many dependencies; making sure all of them are included in the release can be like “finding a diamond in the rough”

Some products are made of components (such as API, data bases, or user interfaces) that may be spread out among different teams. To make sure the right pieces are glued together to produce, at the least, an MVP, release teams must be able to pinpoint what these pieces are and their relationship with each other. This reduces the risk of wasting time on items not needed immediately.

The image shows a flowchart, showing the process from system level libraries to dashboard for all reports. The Budget Reporting section is highlighted, with a legend indicating that its orange box means Feature Enhancement for Next Release.

This image shows the same flowchart as the previous image, but with specific sections highlighted in red to indicate Components needed for successful release.

Some suggested techniques to help with this exercise are:

  • Drawing a dependency graph.
  • Using existing application architecture diagrams.
  • Using a system context diagram.

There is no one best way for listing dependent tasks of a release feature(s)

The image is the same as from the previous section.

An example of a Dependency Graph

One feature flag in hand is worth many release candidates in the bush

At a high level, feature flags accelerate delivery of value.

Disable or enable a feature: A feature behind a flag allows dev teams to disable a feature that may be creating production problems, work on the feature to fix it, and then redeploy. In this way, if a feature has missed proper testing and the defects make their way to production, the entire release does not have to be rolled back – only the troublesome parts.

Testing in production: Some features just need to be tested in a production environment because (a) security testing may not allow production data to be used in UAT, (b) due to budget constraints, all environments cannot be made the same, or (c) . Feature flags can provide an opportunity for testing systems in a live production environment and, in case of problems, turn off the offending features quickly.

At the same time, not thinking through the reason to use them can be problematic.

Don’t forget that a feature flag is only a means to an end: Once they’ve served their purpose, leaving them in the code can be a risk because unintentionally turning them on or off could result in the system behaving in a way it should not.

Don’t use feature flags without thinking about proper application design and their part in it: Some solutions just have the tendency to become the only solution. In environments where continuous delivery and deployment are not mature, development teams may rely on feature flags for many different types of hypothesis testing. All this does is increase code complexity (and a conscientious software engineer should always keep in mind the simple civic duty of “writing code that anyone can read in the developer's absence”) and code maintenance.

Feature flags and performance issues: Depending upon the complexity of the middleware (e.g. in a compute-heavy system), feature flags will be tested for their state, every time. Those are CPU cycles better used elsewhere (and in a cloud-happy world, wasted CPU cycles mean more dollars spent).

2.2 Select the right features

2 hours

Input

  • Business feature demand
  • Documented backlog

Output

  • List of sub-components needed for delivering on business features demanded

Materials

  • Whiteboard/Flip Charts

Participants

  • Product Owner/Product Manager
  • Software Engineering Team
  • Software Testing Team
  • Infrastructure Team
  • Database Engineering Team
  1. List high-level user stories and related requirements for upcoming releases.
  2. From the documented backlog, identify the pieces that are on the critical path for delivery of listed user stories and requirements.
  3. For each identified piece, document:
    • Its dependency
    • Its complexity
    • The clarity of detailed requirement
    • Estimated effort (for activities like requirements analysis, code development, testing)

2.2 Select the right features (cont.)

Once a list is completed, document their dependencies, complexity, clarity of requirements, and effort estimates.

Feature: Budget Report
Item Dependencies Complexity Clarity of Requirements Estimated Effort
UI Library Data Access Layer for Report Low We have release notes and customer support 4 hours
Third-Party UI Charting Data Access Layer for Report Medium We have release notes and customer support 3 hours
Data Analysis Module High Requirements are complete 10 hours
Data Analysis Module CRUD Library for Data Feeds Unknown The need for this module was discovered as part of feature selection Undetermined
... ... ... ... ...

Anything can be turned into a service, including your release capabilities

Reliable and Repeatable.

A scientifically analyzed process or processes that are studied and improved on a continuing basis eventually come to a point where they don’t remain a mystery any longer. The ability to have non-event releases is no different. Once a level of familiarity with possible procedural variabilities is achieved, automating them to become continuous or as-a-service is the next logical step.

Opti-Mate* your release capabilities using the principles of continuous anything.

*Optimization followed by Automation.

Refer to Info-Tech’s Automate Your Software Delivery Lifecycle blueprint for a detailed look at the concept of Opti-Mate.

Continuous anything has three fundamental guiding principles

Continuous anything is:

  1. A Lean approach. The components used must be continually tuned. We must focus on organizing development activities efficiently and reduce lead time of the development cycle by simplifying complex structures, eliminating overheads, and making hand-offs as seamless as possible.
  2. Not a fixed procedure or a set of tools. It cannot be bought and installed. It is an approach that, if properly followed, forces delivery teams to become aware of their deficiencies and then provides guiding principles to eliminate them.
  3. An engineering discipline based on the scientific principle of falsifiability. One failed test in any stage of the delivery cycle is more important as an indicator of a release candidate’s quality than a series of positive test cases.

Robust continuous “anything” requires proficiency in five core practices

A continuous anything evaluation should not be a “one-and-done” event. As part of ongoing improvements, keep evolving it to make it a fundamental component of a strong operational strategy.

Continuous Anything

  • Automate where appropriate
    • Automation is not a silver bullet. All processes are not created equal; and therefore, some are not worthy of being automated.
  • Control system variables
    • Deploying and testing in environments that are apple to apple in comparison reduces the risk of unintended outcomes from production release.
  • Measure process outcomes
    • A process not open to being measured is a process bound to fail. If it can be measured, it should be, and insights found should be used for improving the system.
  • Select smaller features batches
    • Smaller release packages reduce the chances of cognitive load associated with finding root causes for defects and issues that may result as post-production incidents.
  • Reduction of cycle time
    • Identification of waste in each stage of the continuous anything process helps in lowering cost of operations and results in quicker generation of value for stakeholders.

Automation is hard. Period.

Technology teams are always under pressure from business to do more, faster, cheaper, and better. While these are noble intentions, the business doesn’t necessarily understand the technical requirements for making such demands come to life. Amongst the different asks, faster is mostly a higher priority, leaving technology teams to decide if they should choose quality over cost.

Enter: Automation, and kill all birds with one stone (or script).

It’s not that straight forward unfortunately.

Before automating, think through the why of it, and the why cannot only be limited to businesses immediate asks but also the technology team's future performance.

Remember:

  • Automation is only as good as the resulting burden.
  • Many teams look to automate a part of their process, but they often don't realize the manual automation maintenance burden they created.
  • Simple automation tools chained together can sometimes be more complex than a complicated single tool.

Application release management and the continuous anything mindset

Release management requires the establishment of a deployment pipeline for orchestrating software development. The pipeline and its components must have built-in repeatable reliability and the pipeline must be managed using the scientific principles of execute, observe, experience, and improve.

A robust release management framework can be created (or improved upon) using the principles of continuous anything.

Robust Release Management Attributes Map Directly to Principles of a Continuous Anything Mindset
Release Management Attribute Automate Where Appropriate Reduce Cycle Time Control System Variables Select Smaller Feature Batches Measure Process Outcomes
Deployment Pipeline
Repeatable Reliability
Application of Scientific Principles

The deployment pipeline forms the basis of continuous delivery (or deployment, if you are ready for it)

The deployment pipeline applies scientifically rational engineering principles and a Lean approach to software development. In an ideal situation, it should be the only mechanism used for delivering software to the end user.

Good software development relies on the Requirement-Code-Test-Deployment wheel. By making all these functions part of the deployment pipeline, we can:

  • Eliminate waste in these functions by focusing on necessary steps only.
  • Focus on lead time and use data for increasing throughput (i.e. better software faster).
  • Optimize our SDLC overall.

Info-Tech Insight

A deployment pipeline is not the only thing needed for a strong and robust release capability. It is a technical manifestation of the continuous anything mindset where a mistake in the present is used as fodder for success in the future.

Refer to Info-Tech’s Modernize Your SDLC blueprint for a detailed investigation of modern software development techniques.

The deployment pipeline takes the principles of continuous anything and represents it as an integrated tool chain

The image shows a large graphic, that shows the deployment pipeline, from Integration to Delivery to Deployment.

Diagram adapted from Continuous Delivery in the Wild, Pete Hodgson, Published by O'Reilly Media, Inc., 2020

Commit often, commit always!

A typical commit has the following steps:

Compile code →Unit test code →Run static analysis →Check in to source control

The purpose of a commit is to make sure the code needed for a successful release is complete, unit tested, and available. At this point, focus on using fast and efficient* feedback mechanisms and ensure core requirements are satisfying a user story’s acceptance criteria.

  • *Get fast and efficient feedback of code readiness by testing for simple and obvious errors in the code. Leave the more sophisticated and time-consuming test cases (like integration and end-to-end tests) for a later time.
  • The best test cases for the commit stage are unit test cases developed as part of test-driven design.
  • 99% of the test cases will be the unit test cases made by developers, with the remaining being linting, code smell checks, build and compile warnings, etc.

Info-Tech Insight

A commit is a purely technical imitation of the change control board. It allows for decision making by shedding light on questions that can be answered simply and quickly, using unit test case execution and their results as proof.

Continuous integration and commit: A match made in DevOps heaven

Slow and steady wins the race.

As part of a release philosophy, our primary focus should be ensuring each commit is tested and deployment ready (irrespective of the fact that the feature it adds to is not completely ready for deployment). For this to be true, commit often and commit always using the practice of continuous integration.

Continuous Integration (CI)

CI is critical to the commit stage. Its claim to fame has always been as a technology but properly developed CI rules have a deep impact on collaborative team building.

As a catalyst for team building, CI creates the space where different functions (e.g. development, testing, infrastructure, product management) can develop a unified plan for source control. Once the “human” plan has been created, using tools like GitHub, teams can create a code check-in and branching approach that is known to all stakeholders.

Related Info-Tech Research

Build Your BizDevOps Playbook

  • BizDevOps is cultural, not driven by tools. It is about delivering high-quality and valuable releases to stakeholders through collective ownership, continuous collaboration, and team-first behaviors supported by tools.
  • BizDevOps begins with a strong foundation in five key areas. The crux of successful BizDevOps is centered on the strategic adoption and optimization of building great requirements, collaborative practices, iterative delivery, application management, and high-fidelity environments.
  • Teams take STOCK of what it takes to collaborate effectively. Teams and stakeholders must show up, trust the delivery method and people, orchestrate facilitated activities, clearly communicate, and knowledge share every time they collaborate.
  • Bring the right people to the table. BizDevOps brings significant organizational, process, and technology changes to improve delivery effectiveness. Include the key roles in the definition and validation of your BizDevOps vision and practices.
  • Focus on the areas that matter. Review your current circumstances and incorporate the right practices that address your key challenges and blockers to becoming BizDevOps.
  • Build your BizDevOps playbook. Gain a broad understanding of the key plays and practices that make a successful BizDevOps organization. Verify and validate these practices to tailor them to your context. Keep your playbook live.

A collaboratively developed source control strategy helps in cementing successful team cultures

Git Flow versus Git Hub Flow?

Your choice of branching strategy will play an important role in your delivery teams’ ability to be proficient at continuous delivery. The question of which is better is entirely a matter of existing strategy and the maturity of the software development process (especially application architecture and software design philosophy).

The image shows a graphic with a series of connected colour-coded dots, according to categories listed on the left side of the image. The graphic is titled Git Flow.

Pro:

  1. Better suited to distributed teams working on different parts of a feature.

Cons:

  1. Can be harder to manage if there are many branches active together. The image shows a graphic with two large pink dots on the left side of the image, connected to a line of white dots, leading up to a larger white dot. It is titled Git Hub Flow.

Pro:

  1. Ideal for increasing flow from idea to releasable* product.
  2. Supports software-release-as-a-service model.

Cons:

  1. Harder to control quality if upstream software engineering practices are not mature.

*Git Hub Flow does not have release as a concept. For our purpose, “release” refers to deployable software.

Branch management and code review

Source code management and integration is a vital step in continuous integration (and continuous delivery and/or continuous deployment). Prior to determining the strategy that makes most sense for your purposes, investigate the following:

  • How many branches are too many? Has having more branches been a benefit or a challenge?
  • Is the development team ready to try other styles of branch management techniques like trunk based?
  • How is code review being done currently? Has it been a useful step, or does it add more overhead, with minimal benefits?
  • Is the size of our “work in progress” (i.e. features being worked on simultaneously) appropriate for our team size and capabilities? Can we break our features and its associated application architecture into smaller modules and use incremental feature development as part of our development and delivery style?

Incremental Feature Development is an engineering discipline in which a feature is broken down into smaller code parts (e.g. a data access layer, a UI, and a middleware) and each smaller piece is worked on separately. With each code check in, smaller pieces are integrated with the main branch and validated for accuracy and quality. After successful testing, the pieces are deployed and controlled using constructs like feature flags to ensure end users don’t have access to it until the entire feature is ready for use.

Testing changes prior to merging code can be a challenge

A central pillar of successful continuous delivery is that the master branch should always be in a releasable state. Adhering to this principle means that developers wanting to test how the changes will work when integrated with the rest of the system don’t want to risk merging code into the master branch without validating everything in a “safe” environment – an environment which does not introduce the risk of breaking the build on the master branch.

Pre-Commit Integration Testing Strategy

Set up a copy of the master locally and test integrations locally

Set up a stubbed version of external services on local machine

Let developers provision environments for integration testing on demand

Allow developers to connect their local machines to a shared remote environment (e.g. a staging or pre-prod environment with a copy of the entire system)

Allow developers to deploy a version of their code to a staging or pre-prod environment for a quick test (and remove it as soon as the test is finished)

The commit stage ends successfully when all the agreed-upon components needed for the release have been committed

  • Once all the tests are completed as a part of commit, a release candidate is generated.
  • A release candidate should be a deployable package (even if it does not contain all the subcomponents needed for a feature(s) to provide value).
  • The form of the deployable target is confirmed at this point of the commit stage. For example, if the deployable target is a Docker Container image or a .DLL file or a .SQL script, we aim to create it as the successful conclusion of the commit stage.
  • Finally, the deployable target is tagged using a collaboratively determined naming convention and saved in an artifact repository.
  • The artifact repository is where all release candidates ever produced are stored permanently.
  • An artifact repository requires:
    • An allocated place to store the release candidates (e.g. folder, source control)
    • A plan to coordinate the adding, updating, and removing of artifacts (e.g. Shell Scripts to coordinate the behaviors we want) .
    • A protocol for directories, based on file names and time stamps.
    • An index file as a map to identify required versions.

2.3 Fine tune your commit stage

2 hours

Input

  • Current commit stage

Output

  • Target commit stage

Materials

  • Whiteboard/Flip Charts

Participants

  • Software Engineering Team
  • Software Testing Team
  1. Describe your existing commit stage process in detail. Focus on the part each role (Developer, QA, BA, Infrastructure, Product Owner, etc.) plays during this stage.
  2. List pain points and areas that have been identified as needing improvements.
  3. For each identified pain point and potential improvement, document:
    • Decomposition into root causes
    • Improvements that may reduce cycle time
    • Potential for standardization and automation
    • Variables that can be controlled
    • Measures to assess improvements
  4. As part of your aspired-to commit stage, discuss options for pre-code-merge integration testing setup. Don’t forget to refer to the Guiding Principles from 1.2.

2.3 Example: Fine tune your commit stage

The image shows a flow chart at the top, showing the process from start to Release Candidate Ready for Acceptance Test. The lower part of the image shows a chart with Roles listed on the left, and challenges written under particular headings. There are explanatory notes attached to specific sections of the chart.

2.3 Example: Fine tune your commit stage

Challenge Root causes of the challenge Possible improvements Can be automated? Variables that can be controlled How can the change be measured?
Lack of standards
  • Legacy of not having standards
  • Started but then other commitments make it hard to continue
  • Find the easiest place to start using standards (e.g. code writing)
  • IDE can be set up for static analysis
  • Different teams have to agree and use one standard
  • Static analysis metrics
Monolithic legacy products make unit testing hard
  • Legacy standards
  • Lack of time and no push from management
  • Allocate 10% of our time to architecture improvement (make it modular)
  • N/A
  • Different teams have to agree and use one standard
  • N/A
DB migration is not unit tested --- --- --- --- ---
No formal mechanism for testing data-programming changes against production-quality data --- --- --- --- ---

Using the suggested improvements noted, map out the process flow for target commit state.

By the end of this Phase, you should...

  • Understand the concept of releasability.
  • Establish a mechanism for picking the minimum set of features and tasks that are necessary for a successful release candidate.
  • Have a future-state view for the integration stage of the application release management framework.

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com

1-888-670-8889

Phase 3

Fine tune the acceptance and deployment stages.

Define Your Current Situation

1.1: Pain Points in Status Quo

1.2: Guiding Principles

1.3: Metrics

Define Standard Release Criteria

2.1: Understand Criteria for Release

2.2: Pick the Right Features and Tasks

2.3: Integrate

Define Standards for Acceptance and Deployment

3.1: Acceptance

3.2: Deployment

Implement the Strategy

4.1: Build a Plan to Implement the Strategy

This phase will walk you through the following activities:

  • Feature planning
  • Identification of architecture and development processes
  • Determining artifact

This phase involves the following roles:

  • Product Owner
  • Product Managers
  • Application Architect
  • Software Development Teams

Acceptance Testing: The technology team’s version of user acceptance testing

Confidence to Release

The transition from the commit stage to acceptance is the first instance post development where the focus shifts from being purely technical to considering if business needs have been satisfied. At this point, teams should carry out evaluations from the perspective of the external users of the system until we achieve enough confidence that we are ready to release.

The principle set of purposes of the acceptance stage is:

  • Evaluation of code changes from an external user’s perspective.
  • Testing of code changes in as life-like scenarios as possible (where scenarios are typical day-in-the-life-of events under known business constraints and in production-like environments).
  • To achieve a level of confidence (e.g. 80% of high priority, 75% of medium priority, and 75% of low priority features) that the changed code is ready for production.

Info-Tech Insight

Acceptance testing stage is not the same as user acceptance testing, where a human sign-off is needed for stage termination.

Acceptance Stage: Acceptance tests 101

An effective acceptance test:

  • Is written from an external user’s perspective and worded in a way that is reflective of their interactions with the system.
  • Evaluates the system in life-like scenarios (e.g. time of day, traffic and load, blue sky flows, and error conditions).
  • Interacts with the system under test through public interfaces.
  • Focuses on the what of the system, not the how.
  • Is always aligned with the overall strategic approach for testing and quality assurance.

Acceptance tests can act as a kind of whole system super-integration test. If we assemble all the components that represent a deployable unit of software and evaluate them together, if the acceptance tests pass, we know that they are configured appropriately, work together, and can be deployed successfully.

Acceptance Tests: To automate or not to automate? That is always the question

Automation of test cases where possible is always a preferred option, especially where the test cases are part of a regression suite or targeted at assuring features that are simple in nature but may be used heavily in real-life scenarios.

Manual tests are not without merit.

Manual testing may not be an essential component of a robust continuous anything process, but there are aspects of usability and user experience testing that are better examined through the human ability of fuzzy pattern matching.

It is useful for software with a significant UI component, where a broader assessment of the usability of a system is helpful.

Manual testing should not used as one-off exploratory analysis of the system and not be used in lieu of a thorough and rigorous examination of the system.

Related Info-Tech Research

Automate Testing to Get More Done

  • Good automated testing improves development throughput. No matter how quickly you put changes into production, end users will not accept them if they do not meet quality standards. Escaped defects, refactoring, and technical debt can significantly hinder your team’s ability to deliver software on time and on budget. In fact, 65% of organizations saw a reduction of test cycle time and 62% saw reductions in test costs with automated testing (Sogeti, World Quality Report 2020-21).
  • Start automation with unit and functional tests. Automated testing has a sharp learning curve, due to either the technical skills to implement and operate it or the test cases you are asked to automate. Unit tests and functional tests are ideal starting points in your automation journey because of the available tools and knowledge in the industry, the contained nature of the tests you are asked to execute, and the repeated use of the artifacts in more complicated tests (such as performance and integration tests). After all, you want to make sure the application works before stressing it.
  • Automated testing is a cross-functional practice, not a silo. A core component of successful software delivery throughput is recognizing and addressing defects, bugs, and other system issues early and throughout the software development lifecycle (SDLC). This involves having all software delivery roles collaborate on and participate in automated test case design, configure and orchestrate testing tools with other delivery tools, and proactively prepare the necessary test data and environments for test types.

Writing Acceptance Test: The art and the science

Always strive to write acceptance tests in such a way that substituting the system under the test does not invalidate the test procedures. This is possible using the Four-Layered Approach.*

External most layer: The test cases should focus on what the system is expected to do, not how. Ideally, a non-technical person should be able to read the test description and understand what its intention is.

Make payment with reward points

Make payment with credit card

Make payment with bitcoin

Domain-Specific Language: The middleware that connects the what with the how. The test cases will invoke this middleware part. makePayment(amount, mode)

Integrations: Acts as the final interface between the domain specific command and the system under test.

For external integrations, stubs are a reasonable replacement since testing the quality of external systems should not be part of an organization’s scope.

API to rewards database

Stub API to MasterCard

Stub API to Digital Wallet

System Under Test

*Adapted from Continuous Delivery Pipelines: How to Build Better Software Faster, Dave Farley

Related Info-Tech Research

Build a Software Quality Assurance Program

  • Standardize your definition of a product. Come to an organizational agreement of what attributes define a high-quality product. Accommodate both business and IT perspectives in your definition.
  • Clarify the role of QA throughout your delivery pipeline. Indicate where and how QA is involved throughout product delivery. Instill quality-first thinking in each stage of your pipeline to catch defects and issues early.
  • Structure your test design, planning, execution, and communication practices to better support your quality definition and business and IT environments and priorities. Adopt QA good practices to ensure your tests satisfy your criteria for a high-quality and successful product.

Performance testing

A system’s performance has a big part to play in it being considered releasable. At the same time, it is important to be sure of what must be measured and what are acceptable thresholds for good-enough performance.

Performance tests typically cover:

  • A System’s Usability: What is the user’s experience when they are interacting with the system?
  • A System’s Throughput: How quickly and accurately can the system process varying quantities of transactions?
  • A System’s Latency: How long does a user have to wait for a response from the system (i.e. the time difference between when user initiated transaction with the system to when the system responded with a message of success or failure)?
  • System Soak-Testing: Long-running tests to find out if protracted use causes any unforeseen problems. An example of such a test could be mimicking off-hour long running ETL jobs overlapping with business-hour user transactions.

In general, our aim is to establish pass/fail tests for all performance tests.

Performance testing can get skewed by uncontrolled variables

Performance tests may, over time, become complex, touching many moving parts of the system at the same time. It is likely that there are other types of tests running concurrently, which can skew the results of a performance test run.

To ensure reliable performance test results, it is important to control the environmental variables at the moment the performance tests are under way.

For example, before performance testing, investigate:

  • What external factors are likely to influence the system under test? Is it possible to isolate these influences? For example, run a data processing job on the same server at a later point in time. If there is a real possibility of this data processing job running in the same environment as the system in production, it makes sense to not isolate it while performance testing.
  • Are some necessary external factors needed for running an “honest” performance load test missing? If yes, can they be faked?
  • Is there scope for doing more component-level testing?
  • Could we run the performance tests on a separate, dedicated, network?
  • Are we using the same hardware and configuration for performance testing as we have in production?

Smooth data migration is the most talked about activity in release and rightfully so (after all, it’s the new oil)

Release management is not just limited to getting software from the developer's desk to the end users' smart phone, main frame or desktop. In a data-crazy world, the data that systems produce is also evolving and changes with or adds to existing data stores.

Typical data-related changes requiring management are:

  • Change to data models
  • Change to data definition
  • Change to data base components that assist with data processing (like stored procedures and functions)

Data migration and its testing:

  • The most important phase of testing for data migration is to unit test the migration scripts themselves.
  • Use fake data and run these tests as part of the commit stage to gain fast feedback on the quality of the changes.

An approach to data migration testing:

  1. Clone current production data.
  2. Anonymize the production data (if necessary).
  3. Copy the (anonymized) data across to the test systems to use as the basis for migration testing.
  4. Deploy the release candidate.
  5. Run simple smoke tests on post-migration, major use cases.

3.1 Fine tune your acceptance stage

2 hours

Input

  • Current acceptance stage

Output

  • Target acceptance stage

Materials

  • Whiteboard/Flip Charts

Participants

  • Software Engineering Team
  • Software Testing Team
  • Infrastructure and Ops
  • Database Engineering Team
  1. Describe your existing acceptance stage process in detail. Focus on the part each role (developer, QA, BA, Infrastructure, Product Owner, etc.) plays during this stage.
  2. List pain points and areas that have been identified as needing improvements.
  3. For each identified pain point and potential improvement, document:
    • Decomposition into root causes
    • Improvements that may reduce cycle time
    • Potential for standardization and automation
    • Variables that can be controlled
    • Measures to assess improvements
  4. Clearly identify the function that is primarily responsible for the execution of the identified solution.
  5. Think about the variety of test cases that could be included (e.g. performance, non-functional, data integrations with external sources). Refer to Guiding Principles from 1.2.

3.1 Example: Fine tune your acceptance stage

The image shows a flowchart of processes from start to Release Candidate ready for deployment to production. Below that, there is a chart with roles listed in the left-most column, and tasks along the top row. There are explanatory notes added.

3.1 Example: Fine tune your acceptance stage

Challenge Root causes of the challenge Possible improvements Can be automated? Variables that can be controlled How can the change be measured?
Challenge 1
  • Root cause
  • Improvement 1
  • No
  • Improvement 1
  • Metric
Challenge 2
  • Root cause
  • Improvement 2
  • Yes
----
  • Metric
Challenge 3
  • Root cause
  • Improvement 3
  • No
----
  • Metric
---- ---- ---- ---- ---- ----

Using the suggested improvements noted, map out the process flow for target acceptance state.

Deployment: The end of the beginning of a release

This is the final stage of our deployment pipeline. By this point, a release candidate that has been acceptance tested is known and is ready to be put into production.

By this point, the following facts should be TRUE:

  • Production-like environments have been configured and tested at least once.
  • Release candidate has been deployed and tested (smoke test, E2E test, etc.) successfully in pre-production environments.
  • There are no all-hands-on-deck meetings scheduled for the weekend.

Incremental Deployment Strategies

In an ideal world (and we don’t live in one … yet), switching from an old version of the software to a newer version would happen without any interruption of service. It still can (and will) if we manage the deployment by implementing various flavors of phased transitions:

  • Rolling Transition: Both old and new versions are live and traffic from old to new is diverted gradually.
  • Blue/Green: One version of the system is always live and available while the second version is updated. Once the second version is approved for release, it becomes active while the other version is inactivated.
  • Canary Releases: Changes are rolled out to a select group of users and tested. Once confidence increases, the changes are made more widely accessible. This can be done through feature flags.
  • A/B Testing: Deploy two versions of the system into production at the same time. Assess the performance and reaction from users for both and after a period, settle on the one that shows more overall positive reviews.

Deployment strategy

As a part of developing a deployment strategy, consider the following questions:

  • Can our systems be shut down during our deployment?
  • Do we pursue a big-bang deployment? Or attempt incremental deployments?
  • If we want to pursue incremental deployments, does our application architecture support it?
  • Do we need a specific infrastructure need (e.g. cloud versus on-premises versus hybrid)?
  • Does the cost of infrastructure have any impact on our deployment decision?
  • Is there any additional check (manual or otherwise) like audit logs that should be extracted from the deployment task?
  • Is there a rollback plan that has been tested?

3.2 Determine deployment strategy

2 hours

Input

  • Current process and strategy for deployment stage

Output

  • Strategy for release

Materials

  • Whiteboard/Flip Charts

Participants

  • Software Engineering Team
  • Software Testing Team
  • Infrastructure and Ops
  • Database Engineering Team
  • Product Owner/Product Manager

Discuss deployment strategy for upcoming release. Discuss associated risk of each deployment strategy.

  1. Choose an upcoming release that is of typical complexity; stay away from small, patch-level releases or large, irregular ones.
  2. Review the questions on the previous slide and document the answers to each for the release.
  3. Based on the answers, document two potential release strategies the team can take to get the release from development to production.
  4. Document the resource cost in terms of people and time for each strategy.
  5. Brainstorm the risks of each strategy at each step, from a business and technical perspective.
  6. Score the risks in terms of impact and likelihood.
  7. Choose the strategy that best manages costs and risks.

By the end of this Phase, you should...

  • Have a future-state view for the acceptance stage of the application release management framework.
  • Have a future-state view for the deployment stage of the application release management framework.

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com

1-888-670-8889

Phase 4

Plan upcoming initiatives for enhancing the application release management framework.

Define Your Current Situation

1.1: Pain Points in Status Quo

1.2: Guiding Principles

1.3: Metrics

Define Standard Release Criteria

2.1: Understand Criteria for Release

2.2: Pick the Right Features and Tasks

2.3: Integrate

Define Standards for Acceptance and Deployment

3.1: Acceptance

3.2: Deployment

Implement the Strategy

4.1: Build a Plan to Implement the Strategy

This phase will walk you through the following activities:

  • Feature planning
  • Identification of architecture and development processes
  • Determining artifact

This phase involves the following roles:

  • Product Owner
  • Product Managers
  • Application Architect
  • Software Development Teams

4.1 Implement the strategy

2 hours

Input

  • Outcomes of Phases 1 through 3
  • Guiding Principles

Output

  • Roadmap for enhancement to both individual parts and the entire release framework

Materials

  • Whiteboard/Flip Charts

Participants

  • Software Engineering Team
  • Software Testing Team
  • Infrastructure and Ops
  • Database Engineering Team
  • Product Owner/Product Manager
  1. Gather information on your existing and target-state release framework articulated through phases 1 and 3.
  2. Discuss the gaps in between your current and target state.
  3. Stack rank the gaps in order of criticality and map them onto a roadmap; for example:
    • Now, Next, Later
    • Q1, Q2, Q3, Q4
    • 4 months, 8 months, 12 months

4.1 Example: Roadmap

The image shows a roadmap in the style of a Gantt chart, with specific categories on the left and Qs 1-4 listed at the top.

Bibliography

“7 Benefits of Release Management.” Plutora, August 2016. Web.

Adams, Grant. “Patch Management: Change, configuration and release or something more?” Fox IT, March 2007. PDF.

Breston, Daniel. “Change and Release Management: what are they? What’s missing?” The ITSM Review, April 2014. Web.

Chaffey, Dave. “Forecast growth in percentage of online retail / Ecommerce sales 2017 to 2023.” Smart Insights, 1 Nov. 2021. Accessed 5 April 2021.

“Continuous delivery – the ING story: improving time to market with DevOps and continuous delivery.” CA Technologies, December 2014. Web.

“CVE – Common Vulnerabilities and Exposures.” MITRE, n.d. Web.

Elazar, Lihi. “Release management in the age of CD: are you ready for the responsibility?” LinkedIn Pulse, July 2016. Web.

Embracing Innovation in Government Global Trends 2020. Seamless Government, Sept. 2020. Accessed 5 April 2021.

Farley, D. Continuous Delivery Pipelines: How to Build Better Software Faster. Lean Publishing, vol. 1, 2021. Print.

Hammond, Jeffery. “Five Ways To Streamline Release Management.” BrightTALK, 25 June 2013. Webcast.

Hodgson, P. Continuous Delivery in the Wild. O'Reilly Media, vol. 1, 2020. Ebook.

Kim, Gene, Jez Humble, and Nicole Forsgren. Accelerate. IT Revolution Press, 27 March 2018. Print.

Reynolds. Bryan. The State of DevOps 2020: What You Need to Know. Baytech Consulting, 20 Oct. 2019. Accessed 5 April 2021.

“What is DevOps?” Amazon. n.d. Web.

"World Quality Report 2020-21." Sogeti, 2021.

About Info-Tech

Info-Tech Research Group is the world’s fastest-growing information technology research and advisory company, proudly serving over 30,000 IT professionals.

We produce unbiased and highly relevant research to help CIOs and IT leaders make strategic, timely, and well-informed decisions. We partner closely with IT teams to provide everything they need, from actionable tools to analyst guidance, ensuring they deliver measurable results for their organizations.

MEMBER RATING

10.0/10
Overall Impact

$12,599
Average $ Saved

10
Average Days Saved

After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve.

Read what our members are saying

What Is a Blueprint?

A blueprint is designed to be a roadmap, containing a methodology and the tools and templates you need to solve your IT problems.

Each blueprint can be accompanied by a Guided Implementation that provides you access to our world-class analysts to help you get through the project.

Need Extra Help?
Speak With An Analyst

Get the help you need in this 1-phase advisory process. You'll receive 7 touchpoints with our researchers, all included in your membership.

  • Call 1: Identify current pain points with your release management process. If appropriate, rank them in order of most to least disruptive.
  • Call 2: Outline the guiding principles for your application release management framework and brainstorm metrics for it.
  • Call 3: Understand the concept of releasability criteria and determine approach to feature/task selection.
  • Call 4: Continuous Integration
  • Call 5: Continuous Acceptance
  • Call 6: Continuous Deployment
  • Call 7: Roadmap to the Future

Authors

Suneel Ghei

Usman Lakhani

Visit our IT Cost Optimization Center
Over 100 analysts waiting to take your call right now: 1-519-432-3550 x2019