Tools to Manage Technical Debt

A visualization from Squore showing software quality. Source: Vector Informatik 2020.

In an ideal world, developers follow best practices and implement the best possible solution. In practice, this is rarely the case. Technical debt is often accepted to satisfy immediate business needs. However, this debt must be managed in the long term. Except for the most trivial projects, it's difficult to manage technical debt through manual effort. Fortunately, many tools exist for this purpose.

Engineering systems are diverse because of their choice of hardware, software programming language, development methodology, system architecture, test plan, and so on. As a result, technical debt is also rich in variety: architecture to code, documentation to testing. Tools to address them are also varied. A project might need to adopt more than one tool to manage its technical debt.

Discussion

• What features are expected of tools that manage technical debt?

Good technical debt management tools are expected to have these traits:

1. Polyglot: Tools need to analyze the source code. As such, they must support popular programming languages (Java, JavaScript, C/C++, C#, Python, PHP, etc.). This support could be offered as plugins to the main product.
2. Analysis: Analysis must be from different perspectives including maintainability, reliability, portability and efficiency. Approaches can include static code analysis, SCM analysis, and test coverage. Tool should record historical data so that trends can be observed. Tool should show time needed to fix the debts.
3. Reporting: Dashboard should show a high-level summary and project status. Dashboard should be customizable since engineers and business folks are interested in different levels of detail. Visualizations should be clear.
4. Deployment: The tool can be hosted on-premise or in the cloud. For multiuser access, it should be a web application.
5. Flexibility: It should be possible to override default values and configure the tool to suit the project. For example, thresholds to detect duplicated code must be configurable.
6. Logging: We may wish to get insights into the analysis or troubleshoot issues. Tool should therefore collect logs for future study or audit.
• Could you mention some tools to manage technical debt?


Feature support of some technical debt tools. Source: Pavlič and Hliš 2019, table 2.

Some tools only report code metrics, facilitate code reviews or support only a few languages. Others support multiple languages, cover multiple debt factors, perform risk analysis, visualize debt in different ways, and offer a useful dashboard summary. Among the latter are SonarQube, Squore, and Kiuwan.

Bliss, SonarQube, Checkstyle, and Closure Compiler are some tools that help with static code analysis. Designite can detect design smells, such as a class doing things that are not part of its core responsibility (poor cohesion). Jira Software and Hansoft are examples that identify but don't measure technical debt. Jacoco captures test debt.

Among other tools are CAST Application Intelligence Platform, Teamscale, SIG Software Analysis Toolkit, Google CodePro Analytix, Eclipse Metrics, Rational AppScan, CodeXpert, Redmine, Ndepend (Visual Studio plugin), DebtFlag (Eclipse plugin), CLIO, CodeVizard, and FindBugs.

In general, there are more tools for code/test debts than for design/architecture debts.

• How do tools quantify technical debt?


A cost model for calculating technical debt. Source: Parthiban 2019, table XXXIV.

Tools should show metrics that are simple, clear, correct and objective. They should help in decision making. There are many metrics to quantify debt with respect to design, coding, defects, testing, documentation, and other aspects of product development. Tools should therefore measure code duplication, code complexity, test coverage, dependency cycles and coupling, lack of documentation, programming rules violations, and more.

Ideally, a few numbers that express overall product quality will be easier to understand at a high level. Expressing technical debt as in units of person-days points to the effort needed to remove the debt. This is a good measure but it doesn't indicate product quality. Therefore, we can also express technical debt as a ratio. Generally, debt that's higher than 10% needs to be addressed quickly.

Tool should also capture trends. For example, a declining trend in test coverage might suggest that with each iteration the team is adding more features that it's capable of testing.

In general, effort estimates are hard to get right. The suggested approach is to remove technical debt gradually (pay off the principal) with every release.

• What are some approaches that tools use to manage technical debt?


Conceptual model of technical debt to aid debt quantification. Source: Nord 2016.

Given various aspects of managing technical debt, we note the following approaches:

1. Identify: Code analysis, dependency analysis, checklists; compare against an optimal solution.
2. Measure: Use mathematical models, source code metrics, manual estimation by experts; estimate based on debt type; use operational metrics; compare against an optimal solution.
3. Prioritize: Cost-benefit analysis; repay items with high remediation cost or interest payments; select a set of items that maximize returns (portfolio approach).
4. Monitor: Warn when thresholds are reached; track debt dependencies; regular measurements; monitor with respect to defined quality attributes; plot and visualize trends.
5. Document: Record each item in detail with identifier, type, description, principal and interest estimates; probability of interest payment; correlation with other items.
6. Communicate: Summarize in dashboards; record in backlog and schedule with each development cycle; list or visualize items.
7. Prevent: Improve processes; evaluate potential debts during architecture/design phases; create a culture that minimizes debt.
8. Repay: Refactor, rewrite, reengineer (new features); automate routine tasks; fix bugs; improve fault tolerance in areas of debt.
• What are some shortcomings of tools that manage technical debt?

Tools may differ in the way they quantify technical debt. The calculated metrics may be incomplete or even inaccurate. For example, a tool may fail to capture architectural debt, such as when code violates layered architecture.

There's no clear consensus on the dimensions or boundaries of technical debt. For example, some may consider known defects as technical debt. When tools report false positives, dealing with these can be tedious.

Many tools calculate the principal (cost of refactoring) but not the interest (extra cost due to technical debt). The latter is more difficult to quantify. However, it should be noted that the SQALE method has models to estimate both remediation cost (principal) and non-remediation cost (interest).

Milestones

1992

Ward Cunningham coins the term Technical Debt while working on WyCASH+, a financial software written in Smalltalk.

2001

The Agile Manifesto is signed. In the following years, the concept of technical debt is more widely adopted with the growth of the Agile movement.

Feb

2007

The first lines of code are written for the Sonar platform. In November 2008, SonarSource is founded to accelerate development and adoption of the open source Sonar platform. By mid-2010, SonarSource gets regular downloads. In April 2013, SonarSource gets a commercial edition and is renamed to SonarQube.

Sep

2009

SQALE calculation of indices from remediation cost table. Source: Letouzey and Coq 2010, fig. 7.

In a white paper, Letouzey and Coq introduce the SQALE (Software Quality Assessment Based on Lifecycle Expectations) Analysis Model. They note that in qualimetry, any measure of software quality should be precise, objective and sensitive. The SQALE model provides normalized measures and aggregated indices. Since software is a hierarchy of artefacts, the model standardizes aggregation rules that lead to remediation indices. Since SQALE is open source and royalty free, by 2016, it's implemented by many tools.

2014

Multiple studies are published in literature that attempt to understand technical debt and relate it to software quality. A couple of these studies use SonarQube and its plugins for analysis.

Mar

2015

Feature comparison of some technical debt management tools. Source: Li et al. 2015, table 16.

Li et al. bring together different studies and summarize the capabilities of 29 different technical debt management tools. They map them to various attributes: identification, measurement, prioritization, monitoring, repayment, documentation, communication and prevention. Most are able to identify or measure technical debt. In addition, they capture what sort of technical debts these tools capture: code, design (code smells or violations), testing, requirements, architecture, and documentation.

Apr

2015

Jira allows tracking of technical debt items. Source: Radigan 2015.

In a blog post, Atlassian recognizes the importance of technical debt. The note that their Jira software can now list and track technical debt, including when a debt was created and when it's expected to be resolved.

Nov

2018

DeepSource is released. It's a source code analysis tool that can integrate with GitHub. Test coverage, documentation debt, security, and dependency debt are some things it can capture.

Oct

2019

A refactoring suggestion in SonarQube. Source: Oswal 2019, fig. 42.

Oswal studies the support for technical debt in three tools: SonarQube, PMD and Code Analytix. He notes that no single tool gives a holistic view and therefore he uses multiple tools for this analysis. The study looks at the Core Java 8 project plus a web application (Java/JavaScript/HTML/XML). The study addresses software reliability, maintainability and security.