Automation Analysis: Deciding What Is Worth Automating
Most automation projects fail before a single trigger fires. They fail at the selection step, when somebody picks a process because it is annoying rather than because it is a good candidate. Automation analysis is the discipline of scoring work before you build anything: how often it happens, how long it takes, how clearly it can be described as rules, what it costs when it goes wrong, and how stable the surrounding systems are. Get the analysis right and the build is usually easy. Get it wrong and you spend three weeks automating something that runs twice a year and breaks every time a vendor changes a form field.
This guide gives you a scoring model you can run in an afternoon, a worked example, the categories of work that should stay manual, and the practical questions of maintenance that nobody asks until month four.
What automation analysis actually is
Automation analysis is a pre-build evaluation. It sits between "someone complained about a task" and "an engineer or ops person starts wiring things together." Its job is to answer three questions in order:
- Is this task a legitimate automation candidate at all?
- If yes, what is the expected return relative to build and maintenance cost?
- What is the failure mode, and can we live with it?
Notice that only the second question is about time saved. Time saved is the metric everyone reaches for first, and it is the least reliable one, because it ignores maintenance and it ignores the cost of a silent wrong answer. A process that saves four hours a month but produces an incorrect invoice once a quarter is a net loss, and no time-savings spreadsheet will show you that.
The other thing automation analysis does, quietly, is force a written description of the process. About a third of the time, that alone is the deliverable. You sit down to document how the weekly reconciliation works, discover that two people do it differently and one of them has been compensating for a bug in the export, and the correct next step is to fix the export, not to automate around it.
The five factors that decide it
Score each factor from 1 to 5. The factors are frequency, duration, rule clarity, error cost, and stability. Four of them are straightforward. The fifth, error cost, is inverted and is the one people get wrong.
Frequency
How many times does this run in a month? Frequency is the multiplier on everything else. A task that happens 200 times a month tolerates a mediocre build. A task that happens twice a year does not justify any build at all, because you will forget how it works between runs and spend longer debugging it than doing it by hand.
Be careful counting frequency at the right grain. "Monthly close" sounds like frequency 1, but if the close involves reconciling 40 accounts, the automatable unit is the account reconciliation, and frequency is 40.
Duration
How long does one instance take a competent person, including context switching? Include the switching cost honestly. A two-minute task that interrupts deep work three times a day is more expensive than its stopwatch time suggests, because the interruption cost is real even though it is not on the clock.
Duration and frequency together give you raw time exposure. That is your ceiling on return, not your return.
Rule clarity
Can you write the decision logic as if/then statements without using the words "usually," "it depends," or "you just know"? This is the single strongest predictor of whether an automation will survive. Test it by writing the rules down and handing them to someone who has never done the task. If they can execute correctly from the document, rule clarity is high. If they come back with questions you answer by describing a feeling, it is low.
Language models change this factor but do not eliminate it. An AI step can handle fuzzy classification that a rigid rule cannot: sorting inbound email by intent, extracting a delivery date from a paragraph, summarizing a thread. What it cannot do is make a judgment call that has no defensible right answer, and it cannot be held accountable for one. So the modern version of this question is: can the decision be described clearly enough that a careful stranger, or a model with good instructions, would reach the same conclusion you would, and could you tell from the output whether it did?
Error cost
What happens when this runs wrong and nobody notices for a week? This factor is scored inversely: low error cost scores high, because low error cost makes automation safe. Sorting files into folders wrongly is a nuisance. Sending a wrong payment amount, misclassifying a customer's contract tier, or emailing the wrong 400 people is not.
Two separate dimensions live inside error cost, and it helps to name them:
- Severity: how bad is one wrong output?
- Detectability: how quickly and reliably would you notice?
A high-severity, high-detectability error is often acceptable to automate, because the failure surfaces immediately and you fix it. A low-severity, low-detectability error is more dangerous than it looks, because it accumulates. Slowly corrupting a CRM field for eight months is worse than one loud failure.
Stability
How often do the inputs, the systems, or the rules change? Stability is the maintenance tax. If the source is a vendor portal with no API and you scrape a page layout, stability is low no matter how clean the rules are. If the source is a well-versioned API with deprecation notices, stability is high. Regulatory or policy-driven processes sit in between: the systems are stable, but the rules change on a schedule you do not control.
For the mechanics of building sources that hold up over time, automated data collection systems covers the architectural side of stability in more depth than fits here.
The scoring table
Score each factor 1 to 5 using this rubric. Sum for a total out of 25.
| Factor | 1 point | 2 points | 3 points | 4 points | 5 points |
|---|---|---|---|---|---|
| Frequency (runs per month) | Fewer than 1 | 1 to 3 | 4 to 10 | 11 to 50 | More than 50 |
| Duration (per run, incl. context switch) | Under 2 min | 2 to 10 min | 10 to 30 min | 30 to 90 min | Over 90 min |
| Rule clarity | Pure judgment, no written rules possible | Mostly judgment, many exceptions | Rules exist with frequent exceptions | Clear rules, rare exceptions | Fully deterministic, no exceptions |
| Error cost (inverted) | Severe and hard to detect | Severe but obvious immediately | Moderate, caught within days | Minor, caught quickly | Trivial and self-correcting |
| Stability | Inputs or rules change monthly | Change quarterly | Change a few times a year | Change annually | Effectively fixed |
Interpreting the total:
| Total score | Verdict | What to do |
|---|---|---|
| 20 to 25 | Automate now | Highest return per unit of build effort. Start here. |
| 15 to 19 | Automate with guardrails | Build it, but add logging, an alert on failure, and a review step for the first month. |
| 10 to 14 | Partially automate | Automate collection and preparation. Keep the decision with a person. |
| Below 10 | Leave it manual | Or fix the underlying process first, then re-score. |
Two override rules sit on top of the total, and they beat the arithmetic:
Override one: any factor scoring 1 caps the verdict at "partially automate." A 22 out of 25 with error cost at 1 is not a green light. It is a high-volume, high-value process where a silent mistake is expensive and invisible. That is exactly the shape of the automations that cause incidents.
Override two: rule clarity of 1 or 2 means automate the inputs, not the decision. Gather the data, assemble the context, put it in front of the person, and let them decide in ten seconds instead of ten minutes. This is where most of the realistic return lives in knowledge work, and it is chronically underrated because it does not feel like "real" automation.
A worked example
Take weekly pipeline hygiene at a 12-person sales team. Every Monday someone opens the CRM, finds deals with no activity in 14 days, cross-references the last email in the shared inbox, and pings the owner.
- Frequency: 4 runs a month, but roughly 30 deals reviewed per run. Score 5.
- Duration: 45 minutes per run. Score 4.
- Rule clarity: "No activity in 14 days" is deterministic. "Is this actually stalled or did they just reply on WhatsApp" is not. Score 3.
- Error cost: worst case, someone gets a nudge about a deal that is fine. Trivial and self-correcting. Score 5.
- Stability: CRM fields change maybe once a year. Score 4.
Total: 21. Verdict: automate now, with the rule-clarity caveat pushing the judgment call to the human. The automation finds and assembles; the rep decides whether to act.
Where a chat-driven workspace fits
The reason the "automate the inputs, not the decision" pattern is underbuilt is that assembling inputs usually means touching four or five systems, and that historically meant writing integration code. That constraint has loosened. In Skopx, you describe the automation in plain English and it becomes a real workflow: triggers are manual, scheduled with a 15 minute minimum, or webhook, and steps are integration actions, AI steps running on your own provider key, if/else conditions, and field transforms. Skopx connects to nearly 1,000 business tools, and every run is inspectable step by step, which matters more for automation analysis than it sounds, because inspectable runs are how you keep error detectability high.
For the pipeline example, the sentence you would type is:
Every Monday at 8am, find HubSpot deals over $10k with no activity in 14 days, pull the most recent email thread for each from Gmail, and post a summary to the #pipeline Slack channel grouped by deal owner with the last contact date.
What that builds: a scheduled workflow with a HubSpot query step, a Gmail lookup step, an AI summarization step on your key, and a Slack post step. What it returns each Monday is a grouped list with source citations, which the reps act on. Nobody spent 45 minutes assembling it, and nobody handed the judgment call to a machine.
Skopx is a paid product with no free tier: Solo is $5 per month, Team is $16 per seat per month with no seat caps, and both bill from day one. AI costs run through your own provider key with no markup from us. The workflows page has the current capability list, and pricing has the plan details.
The limits are worth knowing before you score a process against them. Workflows are acyclic, capped at 20 steps, and have no human-approval step and no custom code step. If your candidate process genuinely needs a person to click approve mid-run, or needs a bespoke transformation that field mapping cannot express, that is a signal to either simplify the process or use a different tool for that piece. Being honest about this in the analysis phase saves you from a half-built automation you have to abandon.
Categories of work that should stay manual
Some work scores badly no matter how you slice it, and some work scores well but should still stay manual for reasons the rubric does not capture. Both are worth naming explicitly, because "we evaluated it and chose not to" is a legitimate and underused outcome.
Genuinely low-frequency work
Anything running fewer than about a dozen times a year. Build cost dominates, and worse, you lose the muscle memory. When the annual process breaks, nobody remembers how it was configured, and you debug a system you have not looked at in eleven months.
Judgment with no defensible right answer
Performance reviews, hiring decisions, pricing exceptions, deciding which customer complaint gets escalated to the founder. You can automate the data gathering around all of these. The decision itself has no correct answer that can be verified after the fact, which means an automation cannot be evaluated, which means it cannot be improved. Automate the brief, not the call.
Relationship-carrying communication
Anything where the recipient's reaction depends on believing a human wrote it. Condolence notes, apologies after an outage, renewal conversations with an unhappy account. The value is the attention, and automation removes exactly the thing that had value. Drafting assistance is fine. Automated sending is not.
Work that is a symptom of a broken upstream process
If the task exists only because a system exports the wrong format, or because two teams keep separate copies of the same list, automating it makes the breakage permanent and harder to see. Fix the source. The re-keying task disappears, which is a better outcome than automating the re-keying.
High-stakes decisions with low detectability
The override-one category. Financial adjustments, access provisioning, anything touching a system of record that other processes read downstream. If you cannot answer "how would we know within 24 hours that this was wrong," do not automate the write.
Work that is about to change
If a system migration, a policy rewrite, or a reorg is scheduled within two quarters, you are automating a process that is about to stop existing. Wait. Score it after the change lands.
Estimating the real cost, including maintenance
The build estimate is the number people argue about. The maintenance estimate is the number that determines whether the automation is worth it.
A workable rule of thumb: annual maintenance runs somewhere between a fifth and a half of initial build effort, scaled inversely to your stability score. A stability-5 automation over a versioned API may need an hour a year. A stability-1 automation over a scraped portal can consume more time than the manual process it replaced. This is not a fabricated benchmark, it is a planning heuristic, and you should replace it with your own observed numbers after two or three builds.
Add three costs that estimates routinely omit:
- The alerting build. An automation without failure notification is not finished. Budget for it.
- The verification period. The first month should run in parallel with the manual process, or with output review. That is real hours.
- The handoff. Someone other than the builder must be able to fix it. Undocumented automation is a liability with a person-shaped single point of failure.
For the input-side of this, where most fragility actually originates, automatic data collection methods compared walks through which collection approaches carry which maintenance burden.
Running the analysis as a team exercise
Scoring alone produces defensible-looking numbers that reflect one person's assumptions. Do it with two or three people, including at least one who actually performs the task.
A workable agenda for a 90 minute session:
- List candidates without filtering. Twenty to forty items. Anything anyone finds repetitive.
- Score frequency and duration first, quickly. These are factual and fast. Drop anything under a combined 4 out of 10 immediately.
- Score the survivors on clarity, error cost, and stability. This is where the discussion happens. Disagreement about rule clarity is the most informative signal in the room: it usually means the process is not actually standardized.
- Apply the overrides. Explicitly, out loud.
- Pick one from the top band and build it. Not three. One, finished and verified, teaches you more about your own estimates than three half-built ones.
Re-run the exercise quarterly. Scores move. A process that scored 12 becomes a 19 after a system migration makes its inputs stable.
What to measure after you build
Automation analysis does not end at the build. Three measures tell you whether the score was right:
- Failure rate per 100 runs, tracked over time. Rising failure rate means your stability score was optimistic.
- Time from failure to detection. If this is longer than a day, your error-cost score was optimistic too.
- Maintenance hours per quarter, compared against the time saved. This is the only number that tells you whether the automation is actually net positive.
Teams that skip this end up with a portfolio of automations nobody has evaluated since launch, some of which are quietly negative. If you want the broader picture of what analysis tooling can and cannot do without an engineer involved, automated data analysis without coding is a realistic assessment of the boundaries.
Frequently asked questions
How is automation analysis different from process mapping?
Process mapping documents how work flows today. Automation analysis takes that map and evaluates each step for automation fit. Mapping is descriptive, analysis is a decision. You often need the map first, but a map with no scoring step tends to produce automation projects chosen by whoever complained loudest.
What score is high enough to justify building?
Twenty and above is a clear yes. Fifteen to nineteen is a yes with guardrails: logging, failure alerts, and a review period. Below fifteen, the honest answer is usually to automate only the data gathering, or to fix the underlying process. The overrides matter more than the total, and any factor scoring 1 should stop you regardless of the sum.
Does AI change which processes are worth automating?
It changes rule clarity, not the other four factors. Tasks involving unstructured text, classification, or summarization that previously scored 2 on clarity can now score 3 or 4, because a model with clear instructions handles fuzzy inputs. It does not change frequency, duration, error cost, or stability at all. A brittle integration is still brittle when a model is reading its output, and a high-stakes silent failure is still high-stakes.
Should I automate a process I am planning to change?
No. Score it after the change. Automating a process that is being redesigned means building against requirements you know are temporary, and the automation itself becomes an argument against changing the process, which is the wrong incentive.
How do I handle a process with high volume but many exceptions?
Split it. Measure what percentage of instances follow the clean path. If 80 percent are straightforward, automate that 80 percent and route exceptions to a person, with the routing rule written explicitly. The mistake is trying to encode every exception, which produces a fragile automation nobody understands. A clean partial automation with an honest exception queue beats a complete one that fails unpredictably.
What is the most common automation analysis mistake?
Scoring duration generously and stability optimistically at the same time. People overestimate the time a task takes and underestimate how often its inputs change, which inflates the apparent return and hides the maintenance cost. If you correct only one bias, correct stability: go look at how many times that system's export format changed in the last two years before you score it. The methods overview in automated data collection is a useful reality check on how much input drift to expect.
Skopx Team
The Skopx engineering and product team