Leadership should assess AI in software development through delivery outcomes: whether users accept a change, whether it remains stable in the real process, and how quickly problems are resolved. Lines of code, completed tasks, or shorter coding time can be useful operating signals, but they are not business proof of value. Business measurement of AI-assisted development starts by connecting delivery speed with acceptance, quality, and recovery time.
Expert author: Ivan Lozančić Expert reviewer: Tomislav Strugačevac
AI tools can change how teams prepare code, tests, documentation, and error analysis. This may shorten individual work steps. Software delivery also passes through business rules, integrations, data, security checks, user acceptance, and production operation.
If a team completes more development tasks while business users reject changes or incidents increase after release, greater speed is not necessarily progress. The same applies when incidents take longer to resolve because the team must first understand code, decisions, or external dependencies created during development.
Leadership should therefore ask more than, "How much faster was it built?" More useful questions include:
This approach does not reduce the value of AI. It directs assessment towards the outcome the business actually uses.
DORA State of AI-assisted Software Development 2025 examines AI-assisted development in the context of organisation and software delivery. This is useful for leadership because it treats development not as an isolated developer activity, but as part of a system of people, processes, platforms, and ways of working.
Findings from this kind of research require methodological caution. Research can identify patterns and relationships among practices, organisational conditions, and reported outcomes. It does not provide a direct causal formula for a specific organisation. Product type, legacy systems, data quality, regulated processes, integration complexity, and team maturity can materially affect results.
For an ORKA client, DORA is not a performance promise. It is a prompt to examine the organisation's own delivery system: where AI changes the workflow, who reviews the output, and which business consequences should be measured.
The most useful framework connects four levels. Each level needs an owner, evidence, and a decision.
The starting point is not an AI tool but a clearly defined request. The business process owner should describe the problem, users, rules, permitted exceptions, and consequences of error. For an ERP change, this includes official transactions, roles, master data, postings, and integrations affected by the change.
The input owner can be the business process owner, with confirmation from the person accountable for the system. A development team should not independently turn an unclear request into a business rule simply because AI can quickly propose an implementation.
Evidence of input acceptance can be an approved set of scenarios: the normal path, an edge case, a prohibited action, and a rollback procedure.
A team may use AI in selected steps, for example to draft code, suggest tests, explain an existing module, or analyse an error log. Accountability for the decision and review remains with the people delivering the change.
Track the time from an approved request to a change ready for review, but also track coverage of agreed scenarios, code review results, and the status of integration checks. A speed metric without evidence of review encourages misplaced confidence.
For changes with higher business risk, create an explicit record: which AI output was used, what was changed, who reviewed the solution, and on what basis it was approved. This record is not administration for its own sake. It supports later analysis when a deviation occurs.
Business acceptance should confirm the expected outcome, not only the technical implementation. For example, acceptance of a change to a calculation rule does not end when an automated test passes. The team needs to check the expected result of the official transaction, posting, report, user permissions, and integration behaviour when data is missing or delayed.
Define the acceptance measure as a verifiable statement: all pre-approved scenarios have a recorded expected and actual result, and deviations have a process-owner decision. There is no need to invent a baseline or target value before the first measurement. First establish a reliable way to collect data.
After release, track incidents related to the change, time to detect the problem, time to contain impact, and time to a permanent resolution. Separate an urgent rollback from actual root-cause removal. Both data points matter, but they answer different questions.
Delivery quality and stability are not solely a development responsibility. The process owner assesses business impact, the operational team manages incident response, and the development team analyses and removes the technical cause. Leadership should establish clear accountability boundaries before the first serious exception.
Consider a hypothetical change in an ERP integration. With AI assistance, a team quickly prepares a data-mapping update and releases it after basic tests pass. In production, a rare input-data format interrupts processing of some documents. The issue can be rolled back quickly, but analysis takes longer because the decision record and edge-case coverage are unclear.
The conclusion is not that AI caused the problem. The cause may be an incomplete request, a missing scenario, weak review, or unclear accountability. The lesson is different: delivery speed needs to remain connected to input quality, acceptance criteria, and the ability to recover quickly.
One set of metrics will not fit every team. A small internal automation and a change affecting financial records do not carry the same risk or need the same level of review. Too many measures can slow decisions, while too few can hide consequences.
It is also important to avoid using measurement incorrectly. A metric should not become an individual ranking of developers by code volume or closed-task count. That approach can reward volume over understanding, collaboration, and maintainability.
A sound starting question is: which decision becomes better with this measure? If a measure does not help decide on investment, review scope, acceptance, or incident handling, it is unlikely to be a priority.
Before wider use of AI-assisted development, document the following:
If the first step is unclear, start with Business process screening . Organisations seeking to connect daily collaboration, official ERP transactions, and accountability through an operating model can also review the ORKA operating system .