When Good AI Practices Go Wrong, Part 1

Artificial intelligence is rapidly becoming part of everyday work. Employees use general-purpose tools such as ChatGPT, Claude, and Gemini to draft, summarize, research, structure ideas, and support problem-solving. At the same time, AI is being embedded into the systems organizations already use: project-management platforms that recommend priorities, dashboards that identify patterns, recruitment tools that screen CVs, or workflow tools that automate routine tasks.

How do we define AI use case maturity? 

The use of artificial intelligence in organizations is not mature simply because it is advanced, widespread, or fast.

A practice becomes mature when it is aligned with the organization's ability to set direction, measure performance, improve work, manage employee contribution, and sustain the culture in which those activities take place. In a mature environment, AI is introduced to address a defined need, supported by clear ownership and appropriate safeguards, and assessed according to whether it improves the quality of decisions and outcomes.

The question is therefore not only, “Can we use AI for this?” It is also, “Does our organization have the capability to use it well?”

A maturity lens for everyday AI use in the workplace 

This article examines everyday AI use through the lens of organizational maturity.

Its examples concern AI in core work: drafting emails, creating content, supporting project management, monitoring information, screening CVs, organizing tasks, and producing reports. This article is the first of a two-part series on AI use and performance management maturity. Part 1 examines the organizational conditions that determine whether AI initiatives are strategically purposeful, measured against meaningful outcomes, and translated into genuine operational improvement. It focuses on three capabilities: Strategic Planning, Performance Measurement, and Performance Improvement. Part 2 will examine the human system around AI use — Employee Performance Management and Performance Culture: how AI changes individual contribution, performance expectations, recognition, communication, learning, leadership, and professional judgement.

The table below introduces the five capabilities of GPA Unit's integrated performance management maturity model. This first article focuses on the first three; the remaining two will be examined in Part 2.

The examples that follow are drawn from real accounts posted by employees and practitioners in public online forums, describing how AI actually landed in their day-to-day work. 

Strategic Planning Capability

Many AI initiatives fail before implementation begins. They are launched because leaders have seen a compelling demonstration, competitors appear to be adopting similar tools, or the organization feels pressure to show visible progress. In these cases, AI becomes an initiative in search of a problem.

Strategic maturity does not require organizations to predict every future use of AI. It requires them to distinguish between a technology trend and a business priority. This means asking where AI could realistically improve outcomes, what operational constraint it is meant to address, who will own the change, and how its contribution will be assessed.

The following examples show the difference.

1.1. Strategy Foundations: turning an AI demonstration into an organizational ambition 

At low maturity, AI becomes part of the organization’s future narrative before leaders have established what role it should play in the business. 

A compelling demonstration creates an inflated sense of possibility, while the organization’s actual work, constraints, data, and priorities remain insufficiently understood.

“I think in terms of negative impacts, the biggest one is giving people an impression that ‘anything is possible,’ when that's far from the truth.”

“Exec saw a hype video about multi agent systems and how they can just do everything.” 

The problem is not that leaders are interested in new technology. It is that the ambition is disconnected from operational reality. The organization adopts a story about AI before it has identified the business problem, the affected process, the available data, the human expertise required, or the conditions under which the use case would create value.

This creates a strategic gap. Employees receive a broad message that AI is “the future,” but no clear direction on what should change, why it should change, or what good implementation would look like. 

What would raise maturity? The organization should translate AI ambition into a defined strategic question. Rather than asking, “How can we use AI?”, leaders should identify a priority outcome or constraint: for example, reducing response time in a customer-service process, improving the quality of project reporting, or identifying recurring causes of operational delays.

This requires a clear AI position within the organization’s strategic direction: the problem to be addressed, the expected value, the boundaries of use, accountable ownership, and the criteria through which the initiative will be reviewed. The relevant maturity-building practice is Strategy Foundations: ensuring that AI-related ambitions are connected to the organization’s purpose, priorities, and actual operating context.

1.2 Strategy Alignment: preventing AI from becoming a side project 

At low maturity, AI initiatives run alongside the organization’s strategy rather than through it. 

They attract leadership attention, consume specialist capacity, and generate visible activity, but remain weakly connected to a defined objective, delivery plan, or expected business outcome.

“It's a shiny object that captures management's attention that creates a lot of side project work for developers with little to show for in the end.”

Experimentation is not the issue. Organizations need room to test emerging technologies. The problem arises when an AI initiative is treated as an exception to normal planning and prioritization.

In such cases, developers, analysts, or operational teams are asked to support a pilot in addition to their existing commitments. No work is formally deprioritized. No business owner is clearly accountable for the outcome. And no decision has been made about what evidence would justify scaling, redesigning, or stopping the initiative.

The organization then accumulates AI activity without creating AI value.

What would raise maturity? AI initiatives should be managed as part of the organization’s existing portfolio of strategic and operational work. Before a pilot begins, leaders should define the objective it supports, the process it affects, the accountable owner, the capacity required, the expected benefit, and the review point at which it will be continued, redesigned, or stopped.

For example, a team testing an AI-enabled customer-support feature should not be asked to “explore AI” in parallel with its existing workload. The pilot should be linked to a specific service objective, such as reducing resolution time for a defined category of inquiries. It should have an agreed scope, allocated capacity, and a decision rule based on service quality, customer experience, and operational effort.

The relevant maturity-building practice is Strategy Alignment: connecting AI initiatives to organizational priorities, operational plans, resources, and accountable ownership.

1.3 Strategy Formulation: using AI to accelerate research, not replace strategic judgement

High-maturity AI use does not assume that every complex activity should be automated. It distinguishes between work that benefits from speed and scale, and work that still depends on context, challenge, experience, and human judgement.

This distinction is especially important in strategy formulation. 

AI can accelerate research, organize information, identify relevant frameworks, and help teams prepare for strategic discussion. It is far less reliable as a substitute for the work of interpreting a specific organizational context and making choices within it.

“I use AI all of the time, for all sorts of things, and consider myself highly proficient with it, but I have literally never gotten any incremental value out of it as far as novel strategy other than expediting research, despite trying multiple times. I suggest you use it for learning about what frameworks exist and how to apply them in general, and also to expedite research.”

Why is this a high maturity practice? 

The value lies in the boundary being set. AI is used where it can genuinely contribute: accelerating access to information, structuring initial research, and making relevant approaches easier to explore. It is not treated as an authority capable of creating a meaningful strategy without human understanding of the organization, its stakeholders, constraints, trade-offs, and ambitions.

This is a best practice in Strategy Formulation: using AI to improve the quality and speed of strategic research while keeping strategic judgement, decision-making, and ownership with the people accountable for the strategy.

Performance Measurement Capability

AI implementation creates value only when the organization can distinguish adoption activity from improved performance.

This is where performance measurement maturity becomes decisive. An organization may track how many employees use an AI tool, how many workflows have been automated, or how quickly a pilot has been deployed. These measures can be useful. But they do not, on their own, show whether AI has improved the work the organization exists to do.

2.1. Target Setting: when AI adoption becomes the target

At low maturity, implementing AI becomes a target in itself. Adoption is treated as progress even when the tool has not demonstrated that it improves quality, productivity, service, or decision-making. 

The result is a dangerous split between the work that a department is formally responsible for and the work it is increasingly expected to do in support of an AI initiative.

“Earlier in the year, my employer developed an AI tool that was supposed to automate a somewhat minor — and frankly pretty annoying — part of my department's daily routine. We all gave it a try, and when they asked how well it was working, we told them it really didn't, but maybe we could use it for help brainstorming ideas sometimes. Just recently however, we got an iron-clad mandate from on high that every department needs to make more serious strides toward implementing AI into our daily work over the current fiscal year. Every single day, like a quarter of my small department now spends most of their time working on this program — doing evaluations, suggesting fixes, trying to come up with strategies to incorporate it into our workflows. We brought these concerns to management, and they actually told us that it's okay if we don't hit all of our department's productivity targets — you know, the actual work the department was created to do — because getting the AI pilot into shape is a higher priority.”

The organization has replaced a performance question with an adoption target. The relevant question should have been whether the pilot improved a defined part of the department’s work. Instead, success appears to be defined as making the tool work, regardless of the effort it consumes or the effect on the department’s actual responsibilities.

The employee feedback also shows that the organization had already received evidence that the tool was not delivering its intended value. Yet that evidence did not change the decision. The pilot continued to absorb time, create unpaid overtime, and undermine morale, while the department remained accountable—at least implicitly—for its usual operational output.

This is not simply a technology issue. It is a target-setting issue. The organization has not established a clear baseline, a realistic value hypothesis, an acceptable level of resource investment, or a defined threshold for pausing or ending the pilot.

2.2 Performance Measurement Enablers: creating the conditions for an accountable pilot

The target-setting problem in this example is reinforced by weak performance measurement enablers. 

There appears to be no clear governance process through which employees’ feedback can trigger a review, no documented distinction between business-as-usual performance and pilot work, and no transparent communication about how the department’s contribution will be assessed while the experiment is underway.

A mature AI pilot would begin with a defined business problem and a testable value hypothesis. For example: can this tool reduce the time required for a specific routine activity by 20 percent without reducing the quality of the output?

Before deployment, the organization would establish a baseline for the existing process, identify the people and time required for the pilot, and define separate measures for two different questions:

• Does the tool work well enough to justify further investment?
• Is the department continuing to deliver its core operational responsibilities?

The pilot would also require protected capacity, a named business owner, regular review points, and a documented decision rule: scale, redesign, pause, or stop. Employee feedback would be treated as implementation evidence, not as resistance to be overcome.

The relevant maturity-building practices are Target Setting and Performance Measurement Enablers: ensuring that AI adoption is measured against real operational value and governed through clear targets, ownership, feedback loops, and decision criteria.

2.3 Data Gathering: using AI to monitor information at scale 

AI can create clear value when it reduces the manual effort required to gather recurring external or operational information. This includes monitoring media coverage, brand mentions, stakeholder discussions, market developments, or other information sources that would otherwise require employees to search, collect, and consolidate data manually. 

“I recently built an agent for a tech company that monitors their key competitor’s online activity and sends a report on slack once a week. It’s simple, nothing fancy but solves a problem.”

“Marketing, sales and strategy departments get the report via slack, so nothing gets missed and everyone has visibility on the report.”

“This helped me save lots of time by having agents monitor media and mentions.” 

Why is this a high-maturity practice?

The value lies in automating a repetitive data-gathering activity rather than automating the judgement that follows from it. The AI agent scans a defined information environment, based on configured keywords and criteria, while employees retain responsibility for interpreting relevance, identifying patterns, and deciding whether action is required.

The use case becomes mature when its monitoring parameters are clearly defined: which sources are included, which keywords are used, how frequently results are reviewed, who validates them, and how relevant findings are escalated. Without these conditions, AI may simply generate a larger volume of information without improving awareness or decisions.

This is a best practice in Data Gathering: using AI to make recurring information collection faster and more systematic, while keeping human judgement responsible for relevance, interpretation, and action.

Performance Improvement Capability 

AI creates operational value when it improves the way work moves through the organization: how projects are managed, how problems are analysed, how teams learn from what works, and how they translate insights into better processes.

This is different from simply producing work faster. A high-maturity use of AI reduces unnecessary effort while preserving the human judgement needed to prioritize, challenge, adapt, and improve. A low-maturity use may automate activity without improving the process beneath it.

3.1 Initiatives Management: using AI to support delivery, not replace it 

AI can be useful in project management when it supports established ways of working. It can help teams organize information, identify priorities, draft routine communication, summarize meetings, and create initial project structures.  Its value is highest when these functions reduce administrative burden without removing human ownership of delivery. 

“We've been using asana's AI features for roadmaps and task recommendations and it's definitely a step up.”

“The key has been using AI to complement our existing tools, making our processes more efficient rather than replacing them as less time spent on admin, more on delivering real value.”

Why is this a high maturity practice?  AI is integrated into existing tools and workflows, where it helps people focus on the work that requires judgement and coordination rather than on repetitive administrative tasks.

This is a best practice in Initiatives Management: using AI to reduce administrative work within an established project-management system, while project priorities, decisions, accountability, and delivery remain human-led.

A similar practice appears in the use of AI for routine project activities:

“We use AI for assisting project management, such as summarizing meeting notes, drafting an email (to be proofed), or creating an initial roadmap.”

“It's great to be able to allocate the most tedium to AI so I can focus on the more creative side of managing projects and ideas.”

The distinction between assistance and substitution is clear. AI produces a first draft, a summary, or an initial structure; the project manager retains responsibility for reviewing it, applying context, and deciding what happens next.

AI reduces the tedium of preparing the material without taking ownership of the project. 

3.2. Learning and Improvement: using AI to strengthen the learning loop 

AI can support innovation when it helps an organization learn from patterns in its work and turn those insights into better choices. 

In content development, for example, it can help teams analyse what has performed well, generate options for improvement, and reduce the time needed to move from insight to experimentation. 

“AI can help a lot here, both with coming up with content ideas and with creating or improving headlines, hooks, descriptions, images, and even videos.”

“For example, I once built an automation that scraped specific social media accounts and analyzed their posts. Then, based on what performed best, it generated new post ideas for me and suggested how to improve or remix the content that worked.”

“So again, it's usually best to use AI as an assistant, not as the main creator.”

The practice is not simply about generating more content. It creates a learning loop. Existing performance information is used to identify patterns, develop new options, and improve future work.

The strongest aspect of the example is the retained human role. AI helps identify ideas and possible improvements, but it does not determine what the organization should communicate, which ideas fit the brand, or whether a piece of content is appropriate for the intended audience. Those decisions remain dependent on judgement, expertise, and context.

This is a best practice in Learning & Improvement: using AI to turn existing performance signals into faster experimentation and better-informed improvements, while keeping human expertise responsible for quality, relevance, and final decisions.

A simple checklist before introducing AI tools or processes

Before introducing, mandating, or approving an AI use case, leaders should ask the following questions.

A “yes” to these questions does not guarantee that an AI initiative will succeed. It does, however, make it more likely that the organization is using AI to improve performance rather than simply to demonstrate adoption.

What comes next

This first article has focused on whether an organization can introduce and improve AI use responsibly.

Part 2 will examine the human system around AI use. It will focus on Employee Performance Management and Performance Culture: how AI changes what employees are expected to deliver, how their contribution is assessed and recognized, and whether AI strengthens or weakens communication, learning, leadership, trust, and professional judgement.