Jeff Lessin

Product Leader · Enterprise AI Transformation

Measure the work, not the licenses.

I led product managers and workstream leaders across 70 AI pilots at a Fortune 500 enterprise, as part of a consulting engagement, spanning software engineering, data science, product management and cybersecurity. I designed the value framework that decided which pilots were funded, built the evidence system that measured them, and turned the findings into executive decisions on adoption, governance and scale.

70 pilots 4 workstreams 3 → 18 use cases, discovered not specified 1 product shipped from a governance gap
Eight stops. It moves the page with you.
01 · Results

Three numbers, each with its basis on the page.

Every figure below shows what it measures, over what period, and at which rung of the value ladder. A precise figure that cannot be explained is worth less than a defined one that can.

$137M

Savings and efficiencies realized by month six

IdentifiedValidatedRealized

$50M was identified in month one against the value framework. The figure grew as pilots cleared workflow fit, adoption and trust — not as more pilots were counted.

How this was measured

Traced, not estimated. Savings were followed through a continual-monitoring capability I originated, which tracks AI usage from task through department and product to realised value.

The ladder is the discipline. Identified means quantified. Validated means checked. Realized means recognised by the business. Most programmes report the first as if it were the third.

Currency: USD. Period: first six months of the engagement.

88%

Average reduction in task time across defined use cases

Range 74–96%

Task-level, not role-level. An 88% reduction on a task occupying a tenth of a week returns about a tenth of a week — not eight times the output. Converting task savings into enterprise value is what the framework is for, and it is why the realized figure is $137M rather than a multiple of headcount.

How this was measured

Every use case carried its own baseline — the same task performed without AI by a comparable practitioner, set before the pilot started.

The range matters more than the average. The weakest defined use case returned 74%, the strongest 96%. A narrow band across varied work is what distinguishes a measured result from an estimated one.

Debugging counts. Correction and rework time sits inside the AI-assisted figure, not outside it. Excluding it inflates every result.

Validated. Business systems analysts checked the records and refined the methodology where it was producing unreliable data.

79%

Reduction in tech-debt and defect-remediation workload, sustained over six months

A composite measure, and deliberately so: fewer defects reaching review, faster review cycles, and reduced dependence on external QA and engineering capacity. The workload is composite, so the measure is too.

How this was measured

Three contributors. AI in the delivery lifecycle reduced errors at source, made code review materially more efficient, and removed the need for a significant share of external QA and engineering capacity.

Why composite rather than one number. Reporting defect count alone would understate it; reporting hours alone would hide where the hours went. Both moved, and both are part of the workload.

Sustain is the finding. Measured monthly across six months. An early improvement shows a tool works on day one. A held improvement shows the workflow absorbed it.

02 · The mandate

Four enterprise objectives, agreed before a use case existed.

The question was never "where can we use AI." It was "where does the enterprise need materially better outcomes, and can AI change that work without degrading quality." These four answers shaped everything downstream — which pilots ran, what was measured, and what counted as success.

Objective 01

Improve staff productivity and capability

Expand what individuals and teams can accomplish — reducing effort where appropriate, but also enabling higher-value work people could not previously complete as effectively or as quickly.

Capability · Capacity · Learning
Objective 02

Increase delivery velocity and product quality

Accelerate the full product-development lifecycle without trading speed for quality. Measure whether gains survived review, reduced rework, and improved what actually reached customers.

Velocity · Quality · Outcomes
Objective 03

Reduce external-worker spend

Find work being performed through external capacity and test where AI-enabled internal teams could absorb, augment or avoid that demand while preserving the required quality and expertise.

Cost · Capacity · Insourcing
Objective 04

Maximize internal staff engagement

Use adoption to reinforce empowerment, accountability and ownership. Workstream leaders were expected to define success, learn from evidence and own the outcome — not simply adopt a tool.

Engagement · Ownership · Empowerment

The operating principle. Productivity mattered only when it translated into stronger capability, better products, lower structural cost and greater ownership of outcomes. Objective 03 is the one that connects the numbers above: external QA and engineering capacity the organisation no longer needed is both a share of the realized savings and a large part of the reduction in defect and tech-debt workload.

03 · The framework

How value was defined before anything was funded.

Most AI value frameworks are return-on-investment calculators with extra steps. This one is layered rather than listed: it states in advance what will cause a high-return use case to deliver nothing. That is what makes it testable, and it is why it occasionally cost us a pilot people wanted.

Outcome

Accelerate strategy

Speeds an objective the enterprise has already committed to.

Outcome

Deliver enterprise return

Tangible, quantifiable value measured in dollars.

Test

Strategic fit

Does this advance a stated strategic objective, or only a local one?

Test

Cost and value

Cost against benefit in dollars. Return drives enterprise value.

Test

Business and workflow fit

Does it fit how the work actually gets done, so the value is realisable?

Select a prerequisite to see what fails without it.

It started as six flat dimensions.

The first version scored six dimensions side by side, each a peer of the others. It compared unlike use cases well enough, but nothing could be funded from it — six scores and no answer — and the value dimension in particular was too soft to act on. Refining it produced the model above. Each move fixed something specific, and the questions the original rubric asked are still the questions underneath.

01 · Productivity and output velocity
Did effort, elapsed time, throughput or freed capacity materially improve?→ Became the measurementSpeed is an input to value, not value itself. It is still scored on every pilot — but it was removed from the list of things that can make a use case count as valuable.
06 · Cost, value and strategic fit
Did the gain connect to a strategic objective and a real economic outcome?→ Split into two testsHeld together it hid which half a use case failed. A use case can be perfectly aligned to strategy and still not worth funding, and the reverse.
04 · Business and workflow fit
Did the end-to-end workflow improve, or did the friction just move?→ Kept as a testThe one that most often separates identified value from realised value.
02 · Quality and trustworthiness
Did the output survive expert review with equal or less rework?→ Moved beneath the testsStill scored, but no longer traded off against the others. It became a condition on all of them.
03 · Adoption and user experience
Did people use it, trust it enough to act on it, and come back to it?→ Moved beneath the testsSame reason. A use case nobody uses does not score low. It scores zero.
05 · Enterprise readiness and risk
What security, governance, data or approval dependencies emerged?→ Became the foundationIt should never have been a dimension to trade off against return. It became a gate beneath everything else.

The move that mattered most was the first one. In the original rubric, productivity sat beside business value as one of six scores a use case could do well on — so a pilot that saved a lot of time looked like a win whether or not the time ever turned into anything. It is still measured on every pilot. What it lost was the ability to make a use case valuable on its own: a use case now has to clear the cost-and-value test, priced in dollars, before it counts at all. That is what the $137M rests on. Nothing is in that number for being fast.

Where this cost us a workflow. One workflow was technically capable and still failed to land, because users experienced a loss of continuity, context and control they had not been asked about. Capability is not adoption. The response was to evaluate the whole workflow rather than the feature, treat context persistence and trust as product requirements, and pair any tool change with education and a governed transition path.

04 · The calculation

How a measured task became a recognised dollar.

The question every experienced reader asks, and the one most AI programmes answer badly. Five steps, each producing something the next one needs.

1

Measure the task, not the tool

Baseline the task performed without AI by a comparable practitioner, then measure it with AI — elapsed time and active effort, with correction and debugging time counted inside the assisted figure rather than outside it.

Yields74–96% per use case · 88% average
2

Price what the time displaced

Convert the reduction into dollars against the capacity it actually freed: external labour no longer required, internal capacity redeployed, or cost avoided. Frequency carries as much weight as the percentage — a large reduction on rare work is worth less than a modest one on constant work.

YieldsA dollar figure per use caseIdentified
3

Apply the gates

Strategic fit, cost against value in dollars, workflow fit — then the prerequisites. A use case that was not adopted, or whose output people did not trust enough to act on, returns nothing regardless of what step 2 calculated. This is where identified value stops being automatic, and it is the step most programmes skip.

YieldsWhich use cases can actually convertValidated
4

Trace real usage, not projected usage

The continual-monitoring capability follows actual usage by department, use case and product through to value. A projection multiplies an assumption by a headcount. This follows what people did.

YieldsAttribution that survives review
5

Have the business recognise it

Finance or the business owner confirms the figure against their own numbers. Nothing moves to the top rung on the strength of the model alone.

Yields$137M at month sixRealized

The gap between step 2 and step 5 is where most AI programmes lose their case. Identified value is arithmetic — a percentage multiplied by a rate. Realized value is that same arithmetic after it has survived adoption, trust, workflow fit and a finance review. Reporting the first as though it were the second is the single most common failure in this work, and the whole framework exists to keep them apart.

05 · The operating model

Four stages, and the framework does different work in each.

The executive decision cadence

Task-level evidence only matters if leaders can act on it. Every cycle compressed findings into the same three questions — which kept uncertainty visible instead of rounding it away, and meant no readout could end without a decision.

What we learned

Concrete progress, evidence, failure mode or shift since the prior cycle.

Why it matters

The implication for cost, speed, quality, risk, customer value or scale.

What happens next

Expand, modify, control, investigate, stop, or resolve a dependency.

06 · Decisions

Four calls I made, and what each one cost.

A framework with no visible decision behind it is administration. These are the four that changed the shape of the programme.

01Four workstreams, four competing views of what AI was for

Align objectives across workstreams before identifying a single use case

›

Establish cross-workstream alignment on strategic objectives first. Only then work with workstream leads to identify high-value use cases, ranked by hours, spend, rework time, delivery bottlenecks and where technical debt concentrated. Then map each use case to an objective, and derive the success metrics from that mapping.

Most programmes run this backwards: collect use cases, then retrofit a strategic justification. That produces a portfolio that cannot be measured, because a use case with no objective behind it has no definition of success — only a definition of activity.

The trade-offIt is slower to start, and it means telling workstream leads who arrive with a favoured use case that the answer is not yet. I spent political capital on a delay that produced nothing visible for weeks.
What it producedA use-case-to-objective map that defined what was measured, and an executive dashboard built on that structure — which gave leadership somewhere to push back on the framework before it was used to judge their teams. That part was deliberate, and it is why it landed.
02Analysis paralysis — an open loop of what about this, why don't we

Constrain the comparison set, and open the discovery layer instead

›

Restrict the pilot to a limited set of platforms, and require that all participants test the same specified use cases. No open-ended exploration in the measurement layer.

Unstructured exploration produces anecdotes, and leadership cannot decide on anecdotes. The constraint was never about limiting what people could try — it was about making results comparable across 70 pilots and four workstreams.

The second half mattered more. I expected additional use cases to emerge and did not want the constraint to suppress them, so the constraint went on the measurement layer and the openness went into discovery: twice-weekly working sessions with pilot participants, where emergent use cases were surfaced, evaluated, and added to the measured set if they cleared the bar.

The trade-offSaying no, repeatedly, to a large and opinionated organisation — and accepting that a limited platform set might miss a better option.
What it producedThe initial three use cases became eighteen, all fifteen of the additions surfaced through the working sessions rather than specified up front. Constraint at the measurement layer is what made the discovery layer safe to open.
The one I would lead with in an interview
03Cybersecurity was not comfortable scaling agentic systems

Halt an agentic tooling track on the evidence

›

Neither overrule the objection nor let it block the programme. Collect the data and let the evidence decide. The evidence supported the concern, and one agentic tooling track was halted.

This is the foundation layer of the framework, demonstrated in practice. The framework says security posture sits beneath everything, and that layer is in the framework because of this. A framework whose author can point to the moment it cost him something is a framework that was actually used rather than decorated with.

The trade-offKilling momentum on tooling people wanted, and absorbing the disappointment of the teams running it.
What it producedA precedent that the gates were real. Every subsequent pilot was designed knowing a bad result would stop it.
04AI usage could not be traced from activity through to value

Turn a governance gap into a product

›

Spin up a new product for continual monitoring of AI systems across the organisation — tracing usage across use cases, departments and the end-to-end product, through to return. It is still running and still improving.

This is the gap that keeps most enterprise AI programmes from proving anything. Teams report productivity gains; finance cannot find them in the results. Without attribution, a portfolio is a collection of opinions about value.

What it producedThe provenance behind the realized figure. The savings are a system output traced through usage, not a spreadsheet estimate assembled after the fact — and the capability outlived the engagement that created it.
Identified the need, specified it, shipped it, still in operation
07 · What we found

AI did not remove the bottleneck. It moved it.

This was the most useful thing the programme learned, and none of it was what anyone set out to measure. Select a constraint to see what caused it, what it blocked, and what was built in response.

Six constraints appeared repeatedly around accelerated work, across different workstreams and different tools. Because they recurred, they were treated as properties of the system rather than as isolated tool problems — which is what turned them into an operating model instead of a backlog of complaints.

Governance is the one worth designing for rather than tolerating. Approval queues form because trust gets granted at the level of a whole connection instead of a specific task, which is why E8 and C8 in the library are the remediation and not an afterthought.

08 · The standard

Nothing counts as a use case until it has all ten.

"AI-assisted software delivery" is a capability category, not a use case. A use case is something a named person does on a Tuesday, with a defined input, a defined output, and a baseline it can be compared against. This standard is what gives the 88% a denominator.

Business objectiveReduce cycle time while preserving quality
User / roleTechnical product owner
InputDiscovery notes, backlog context, architecture constraints
AI-assisted taskProduce draft stories and acceptance criteria
Defined outputReview-ready backlog items
BaselineThe same task without AI, by a comparable practitioner
Productivity measureElapsed time and active effort, debugging included
Quality measureCorrections, completeness, downstream clarification requests
Business measureReady-for-development throughput
Decision thresholdContinue, modify, expand or stop — numbers fixed before the pilot

AI assistance will reduce [productivity measure] without increasing [quality measure].

09 · The evidence system

What made the numbers trustworthy enough to report.

Adoption data tells you whether people used AI. It cannot tell you how value was created, where quality broke, or what to test next. The answer was structured task-level evidence, challenged by analysts before anyone saw a summary.

01 · Capture

Turn a work session into task records

At the end of a session the work was converted into discrete task-level records against a fixed schema. Not code volume, not licence activity — what the person was trying to do and what it cost them.

task_descriptionWhat the user actually tried to accomplish
completion_statusSuccessful · partial · unsuccessful
assisted_effortActive time, including retries and corrections
baseline_estimateComparable effort without AI
quality_evidenceCorrections, first-pass success, rework
failure_modeWhy the result failed or needed correction
02 · Validate

Have analysts challenge the record

Business systems analysts reviewed the logs, found inconsistencies, and refined the extraction method rather than treating generated records as ground truth. The failure mode field is the one that earns its place here — a record that cannot say why something went wrong is a record nobody should trust.

Source preservationSession evidence stays retrievable
Schema validationRequired fields, allowed values
Analyst reviewChallenge inferred or ambiguous fields
Method refinementUpdate the instructions when errors recur
03 · Consolidate

Build an analysis layer that can be audited

Validated records were consolidated with identity, duplicate handling and quality flags, so any executive summary could be traced back to the task record behind it.

Unique identitySession and task keys
DeduplicationPrevent double-counted work
Quality flagsSeparate strong evidence from weak
AuditabilityTrace a summary back to its record
04 · Evaluate

Assess against the refined value model, in its own order

Refining the value definition made the same evidence legible across four very different workstreams, and kept the focus on what the organisation actually valued rather than on what moved fastest. The six questions are still asked. They are no longer peers — the lower layers disqualify, so they are checked first.

FoundationSecurity postureCan this scale at all, or does the risk position stop it here?
PrerequisiteQuality and trustworthinessDid the output survive expert review with equal or less rework?
PrerequisiteAdoption and user experienceDid people use it, trust it enough to act on it, and come back?
TestStrategic fitDoes it advance one of the four stated enterprise objectives?
TestCost and valueCost against benefit in USD, priced on the capacity it freed.
TestBusiness and workflow fitDid the end-to-end workflow improve, or did friction just move?

Then, and only then Does it accelerate strategy, deliver enterprise return, or both? Productivity is measured throughout and is not one of the answers.

The system became a discovery loop, not a reporting layer. Patterns in the validated logs surfaced high-value work nobody had anticipated during pilot planning, and those opportunities were folded into later phases. That is the mechanism behind three use cases becoming eighteen — the measurement did not just record the portfolio, it found it.

10 · What it became

The measurement system found a gap. I turned it into a product.

Everything above describes a programme that ended. This one did not.

In operation · still improving

Continual monitoring for AI systems across the enterprise

The pilots kept surfacing the same structural problem: the organisation could see that AI was being used and could see that teams reported gains, but it could not follow either one through to a business result. Usage lived in one place, outcomes in another, and nothing joined them. Every claim about value was therefore an argument rather than a fact.

I identified the gap, specified the product, and stood it up. It traces AI usage continuously — from the work itself, through the use case and the department, across the end-to-end product, to realised value.

SignalUsage in the work
ResolveTask and use case
AttributeTeam and department
FollowEnd-to-end product
LandRealised value
Why this is the one that matters

This is the gap that keeps most enterprise AI programmes from proving anything. Teams report productivity; finance cannot find it in the results; the programme is defended on faith until the budget cycle ends it. The chain above is why the figure in section 01 is a system output rather than a spreadsheet estimate — and it is the piece I would build first at a new organisation, because without it every other measurement is an assertion.

11 · The library

The use-case library I bring to a client.

Thirty-three candidate use cases in the format above, across four workstreams plus the three that belong to none of them. This is the menu a new engagement starts from — the shape of the work before a single tool is chosen.

Two of them (E8 and C8) address the problem enterprises are hitting right now: third-party agent tooling that cannot be inspected, whose capabilities change without notice, and which is granted trust at the level of the whole connection rather than the specific task. One inventories the surface. The other brokers it.

12 · What I would bring

The method, stated so it transfers.

01

Explore against stated objectives

Start with what the enterprise has already committed to, not with a tool. Screen for strategic fit before anyone builds a business case.

02

Pilot as experiment

Define the baseline, the quality measure and the stop threshold before the pilot starts. A use case missing any of the ten elements is a capability category wearing a use case's clothes.

03

Recommend on validated evidence

Task-level measurement, checked by analysts before it reaches executives. Assess workflow fit and adoption alongside return, because the prerequisites are where value leaks.

04

Decide, and record the rung

Distinguish identified from validated from realized, every time. Convert findings into an operating model rather than a report.

↻

Underneath all four: manage outcomes, not output

Track where the bottleneck moves as speed increases. It does not disappear — it relocates, and usually somewhere a tool cannot reach.

On confidentiality

This engagement is covered by a non-disclosure agreement. Nothing on this page identifies the client, describes their systems or workflows, or reproduces their materials. Every framework shown here is my own and has been redrawn from scratch for this page.

I am happy to discuss how the measurement system worked in as much detail as you like. I will not tell you who it was built for — which is the same answer you would want me to give about your programme afterwards.

13 · Next

The method transfers. The question is what you are trying to measure.

If you are standing up an AI programme, or you have one running that cannot yet prove what it returned, I would like to hear what you are up against. I am open to product leadership roles with enterprises outside the United States.

LocationModi'in, Israel Phone+972 53-274-2800 EmailJeffLessin@gmail.com LinkedIn/in/jefflessin