SEO Competitor Analysis Without Enterprise Pricing

You can run most of a competitive SEO program with public Google APIs, a sitemap diff, and about an hour a week, which means the expensive seats sold by mainstream seo competitor analysis tools are optional rather than mandatory. The paid platforms are buying you three things: a keyword index you did not have to build, a rank tracker you did not have to schedule, and a backlink crawl you cannot reproduce. Everything else in a competitive audit comes from data your competitors publish themselves or from datasets Google gives away for free.

This article walks the free path first, marks the exact points where it breaks, and shows what a small automation layer replaces. The goal is not to argue that Ahrefs and Semrush are worthless. The goal is to make sure you are paying for the parts that are genuinely hard to get, not for a dashboard wrapper around sitemap.xml.

What do seo competitor analysis tools actually measure?

Strip the branding away and nearly all seo competitor analysis tools are assembling the same five layers. Knowing which layer you are looking at tells you whether it can be reproduced for free.

The first three layers are free and always have been. The fourth is where free tooling gets thin, though you can get further than most people expect by combining your own Search Console data with manual SERP checks. The fifth is the one legitimate reason to buy a seat, and even then most teams need it quarterly, not daily.

A useful reframe: competitor analysis is not a keyword report. It is change detection. The number that matters is not "they rank for 4,000 keywords." It is "they shipped nine pages about procurement workflows in the last three weeks and rewrote their pricing tiers." That second sentence is actionable, arrives from free sources, and is exactly what a $200 monthly seat rarely surfaces without you digging for it.

How do you diff a competitor's sitemap without paying for a crawler?

Start at https://competitor.com/robots.txt. Almost every site lists its sitemap location there, and larger sites list a sitemap index that points to child sitemaps split by content type. That split is itself intelligence: separate sitemaps for /blog/, /customers/, /integrations/, and /docs/ tell you how the company thinks about its own content estate before you read a single page.

Fetch the sitemap, extract the <loc> values, sort them, and save the file with a date in the name. Next week, fetch it again and run a plain diff. Three signals fall out immediately:

New URLs. These are the pages the competitor decided were worth building. A cluster of new URLs sharing a path prefix is a campaign, not a coincidence. If six new pages appear under /integrations/ in a fortnight, they are chasing integration keywords, and you now know that a month before it shows up in any rank tracker.

Removed URLs. Deletions are underrated. A competitor pruning fifty thin pages is running a content consolidation, and the pages they kept tell you which topics they believe convert. Deleted pricing or feature pages often precede a repositioning.

Changed lastmod timestamps. Many sitemaps carry a lastmod value per URL. When an old page's timestamp jumps, someone rewrote it. Refreshes of existing pages usually mean the page was losing ground, which flags a query where the SERP is currently in motion and therefore winnable.

Two caveats keep this honest. First, lastmod is self reported and some CMS platforms bump it on every deploy, which makes the whole file look like it changed. Test one competitor for a couple of weeks before you trust the field. Second, sitemaps are a claim about what a site wants indexed, not proof of what is indexed. A URL in a sitemap can still be noindexed. Spot check with a live fetch when the diff surprises you.

If you want to see what a page looked like before the rewrite, the Internet Archive publishes a free CDX API that returns every capture of a URL as a list. Pull the capture from before the lastmod change, pull the current version, and diff the visible text. That is how you find out whether a competitor swapped their headline claim, dropped a compliance badge, or changed a price.

Can you check a competitor's Core Web Vitals for free?

Yes, and this surprises people. Google publishes the Chrome User Experience Report, a dataset of real-user performance metrics collected from opted-in Chrome users, and it covers any origin with enough traffic to clear the anonymity threshold. Your competitors are in it. You do not need their analytics access, and there is no scraping involved.

There are two distinct measurements and mixing them up is the most common mistake in performance competitor analysis:

Lab data comes from Lighthouse, usually through the PageSpeed Insights API. It runs a page on a simulated device with throttled network conditions and returns a score plus a diagnostic list. It is repeatable and great for debugging, but it is a simulation of one device, not a description of your visitors.

Field data comes from CrUX. It is aggregated from actual page loads by actual people, reported as a 28 day rolling distribution. This is what Google uses for the page experience signal, and it is the number that reflects reality on cheap Android phones and hotel wifi.

Query the CrUX API for a competitor's origin and you get Largest Contentful Paint, Interaction to Next Paint, and Cumulative Layout Shift as percentile distributions. Compare origin level data first, since it covers the whole site and rarely hits the anonymity threshold. URL level data is available for high traffic pages, which usually means their homepage and their most popular articles.

The competitive read is straightforward. If a competitor outranks you on a query and their field LCP sits comfortably under the good threshold while yours does not, performance is one plausible part of the gap. If their field data is worse than yours and they still outrank you, performance is not your problem and you should stop optimising images and go fix your content. That second outcome is the more valuable one, because it saves you a quarter of engineering time.

Both APIs are free with a Google Cloud API key, and both have published per-minute and per-day quotas that Google adjusts over time. Check the current numbers in the API docs before you write a loop over a hundred origins. If you are wiring either one up yourself, the mechanics are covered in our PageSpeed Insights API guide and in Core Web Vitals monitoring, which explains why field and lab numbers disagree and which one to act on.

How do you measure SERP overlap without a rank tracker?

This is where the free path gets genuinely harder, so be precise about what you need. Most teams do not need a full rank index. They need to answer one question: on the queries that already send us traffic, who else is on the page, and are we gaining or losing?

Start with data you already own. The Search Console API returns your impressions, clicks, average position, and click-through rate per query and per page, going back sixteen months. That is a complete map of your own footprint, for free, with no sampling. Export it and sort by impressions where position sits between 5 and 20. Those are queries where Google already considers you relevant but not the best answer. They are your realistic competitive battleground, and there are usually fewer than a hundred of them.

Now take that shortlist and check the SERPs manually. A hundred queries, checked once a month, is roughly two hours of work. Record the top five results per query in a spreadsheet with the date. After three months you have a rank history for the queries that actually matter to your business, built from primary observation, with no subscription. It is narrower than a commercial rank tracker and it is more accurate for your specific set, because commercial trackers sample from a fixed location and device profile that may not match your buyers.

To find the overlap set, count how often each competing domain appears across your shortlist. A domain that shows up on sixty of your hundred queries is your real SEO competitor even if the sales team has never heard of them. A domain that shows up twice is noise. This exercise regularly reveals that the company you benchmark against commercially is not the one taking your search traffic.

The honest limitation: this tells you nothing about queries where you have zero impressions. Discovering entirely new keyword territory is what a keyword index is for, and no amount of clever free tooling replaces it. If that is your bottleneck, buy a month of a paid tool, export the gap analysis, cancel, and work the list. Keyword universes do not change fast enough to justify twelve months of billing. The Search Console side of this workflow is detailed in our Search Console API guide.

What does a competitor's pricing page tell you that their blog does not?

Pricing pages are the most information dense page on any competitor's site and the least monitored. They change rarely, and when they change, something material happened inside the company.

Track four elements and diff them monthly:

Tier names and counts. Adding a tier below the entry price signals downmarket pressure. Adding one above signals a move to enterprise sales. Collapsing three tiers into two usually means the middle one was not converting.

The price itself. Obvious, but also record what the price is per. A shift from per seat to per usage, or the arrival of a credit system, changes the entire competitive comparison and is easy to miss if you only screenshot the number.

The feature gating line. Which capability sits behind which tier tells you what they believe is valuable. When a feature moves from the top tier down to the middle, it has become table stakes in the category. When it moves up, they found a willingness to pay.

The words around the numbers. Compliance claims, support commitments, and contract language change quietly. A vendor that quietly removes a compliance badge has almost certainly had an audit lapse, and that is a fact your sales team can use tomorrow.

Pair this with their careers page. Job postings are a public roadmap. Three backend hires mentioning a specific database, or a first data engineer, tells you what is being built six months before it ships.

Where does the free path stop working?

Being clear about the ceiling matters more than defending the approach. Four things break down.

Backlink discovery. You cannot build a link index. Anyone claiming a free equivalent is either sampling a tiny public dataset or making it up. If link building is your primary channel, pay for a crawl based tool and treat it as a real cost.

Historical keyword data. Free methods start collecting from the day you start. There is no way to ask what a competitor ranked for two years ago. Paid tools have that archive and it is genuinely theirs.

Scale. Twenty competitors checked manually is not a workflow, it is a second job. The free path holds up beautifully for three to five competitors and collapses past that.

Consistency. This is the real killer. Manual competitive analysis gets done enthusiastically for three weeks and then stops the first time a launch week gets busy. Change detection only produces value if it runs on a schedule, because the whole point is comparing this week to last week. A gap in the record destroys the diff.

The first three limits are structural. The fourth is a scheduling problem, and scheduling problems are the ones worth automating.

What does this look like when it runs on a schedule?

The practical fix is to turn the manual routine into a recurring job so the record never has a hole in it. That is what Skopx does for this workflow. Site Health pulls Lighthouse scores through the PageSpeed Insights API, real-user Core Web Vitals from CrUX, and Search Console performance, then runs an in-house on-page audit that returns a 0 to 100 score with a specific fix list rather than a vague grade. Competitor pulse handles the change detection side: sitemap diffs and pricing-page diffs, reported as what changed since the last check.

The AI visibility side has become part of competitive analysis whether or not you planned for it. Skopx generates buyer-intent prompts from your own site, runs them through search-grounded AI, and reports share of voice alongside citation gaps, meaning the specific prompts where a competitor gets named and you do not. That is the same overlap logic as a SERP check, applied to the surface where a growing share of research now happens. There is also a community openings feed that surfaces live Reddit and Hacker News threads on your topics, which is where competitor comparisons get made in public. If that layer is new to you, start with our guide to measuring AI visibility and the generative engine optimization guide.

Pricing is Solo at $5 a month and Team at $16 per seat a month, which sits below what most competitive intelligence platforms charge for a single seat. AI work runs on your own key with zero markup, or on the allowance included with your plan. On security posture, the accurate statement is SOC 2 controls in place. We do not claim certification, HIPAA compliance, or a contractual SLA, and you should be suspicious of any vendor whose competitive-intelligence marketing is vaguer about that than about their feature list.

If you want the checks that apply to your own site rather than your competitors', what to look for in an SEO audit tool covers the evaluation criteria, and brand mentions monitoring in the AI era covers the tracking side.

What does a weekly competitor routine actually look like?

Here is a schedule that survives contact with a real week. Total time is under ninety minutes.

That last row matters more than it looks. Competitor sets go stale. The company that mattered eighteen months ago may have pivoted out of your category entirely while a startup you have never heard of quietly took three of your rankings. Rebuild the list from your own SERP observations every quarter rather than from memory or from the sales team's list.

Two rules make the routine stick. Write findings as sentences, not screenshots, because a sentence like "they moved SSO from Enterprise to Team on the 4th" is searchable next quarter and a screenshot is not. And attach an action to every finding or delete it. Competitive intelligence that does not change what you build is a hobby.

Frequently Asked Questions

Are free seo competitor analysis tools good enough for a small team?

For most small teams, yes, with one exception. Sitemap diffs, CrUX field data, Lighthouse runs, pricing page monitoring, and your own Search Console data cover content strategy, technical benchmarking, and positioning completely. The exception is backlink analysis, which requires a proprietary web crawl that no free tool can substitute for. If links are your main growth channel, budget for a paid tool. If content and technical SEO are your focus, the free stack genuinely holds.

How often should I run a competitor analysis?

Sitemap diffs weekly, because content ships weekly and you want to see campaigns forming. Pricing and positioning monthly, because they change rarely and a monthly cadence catches everything. Rank and SERP overlap monthly, since daily rank fluctuation is mostly noise and reacting to it wastes time. Backlink gap analysis quarterly. The cadence matters less than the consistency, since every one of these methods works by comparison against your own previous record.

Can I see a competitor's actual traffic numbers?

No, and any tool showing you a precise traffic figure for a site you do not own is showing you a model, not a measurement. Those estimates come from clickstream panels extrapolated to the whole web, and their accuracy varies enormously by site size and geography. Use them for rough ordering, never for planning. What you can measure exactly is their content velocity, their published pricing, and their real-user performance data through CrUX, all of which are facts rather than estimates.

What is the single highest value free competitor check?

The weekly sitemap diff. It takes ten minutes, requires no API key, and tells you what a competitor decided to invest in before that investment shows up in any ranking. Rankings are a lagging indicator by definition, since a page has to be published, crawled, indexed, and evaluated before it moves. New URLs are the leading indicator, and they are sitting in a public file.

Does competitor analysis change now that people search through AI assistants?

The method transfers, the surface changes. Instead of asking which competitor appears in the top five for a query, you ask which competitors get named and cited when an assistant answers a buyer-intent question. The overlap logic is identical: build a prompt set from questions your buyers actually ask, record which brands are cited, and track the gap over time. The practical difference is that a cited source can influence an answer without the user ever visiting the page, which makes citation share the metric rather than clicks. Our article on what changes with LLM SEO covers the mechanics in more depth.

Should I track twenty competitors or five?

Five, chosen from your own SERP data rather than from your sales battlecards. Tracking twenty produces a report nobody reads and a routine nobody sustains. Count how often each domain appears across your top hundred queries, take the five that appear most, and revisit the list quarterly. The domains taking your search traffic are frequently not the ones taking your deals, and the SEO competitor set should be built from search evidence.