Duplicate content is the same or very similar content reachable at more than one URL. Google does not describe it as a penalty. The real cost is that search engines must pick one version, and links and signals can split across copies. You fix it by choosing one preferred URL and pointing the others at it.
Is duplicate content a penalty?
Google's guide to consolidating duplicate URLs treats the topic as a question of which URL is the preferred, or canonical, version. It says that if you do not specify one, Google will identify which version it considers best to show users. Nothing on that page describes a punishment for having copies.
That is good news, but it is not the same as "no problem." Google lists the reasons to consolidate: show people your preferred URL, combine signals such as links into one URL, make metrics for a piece of content easier to read, and avoid wasting crawling on copies. Each of those is a loss you can avoid.
Where does duplicate content come from?
Most small sites do not copy pages on purpose. Duplicates appear by accident:
- Protocol and host variants.
http://andhttps://, with and withoutwww. - Trailing slashes.
/pricingand/pricing/both load. - Tracking and filter parameters.
?utm_source=...,?sort=priceand similar add new URLs for the same page. - Printer, AMP-style or mobile copies that live at their own address.
- Syndicated or republished posts that appear on your site and elsewhere.
- Near-duplicate pages you wrote. Several location or product pages that differ only by a name, or two posts answering the same question.
The first four are technical and fixable in an afternoon. The last two are content decisions, and they need a different fix.
How do you find duplicate content?
- Search Google for a distinctive sentence from the page in quotation marks, with
site:yourdomain.comin front. More than one result means more than one URL carries it. - Open the page indexing report in Search Console and look for statuses that mention a duplicate or an alternate page with a proper canonical tag. Our guide to checking if a page is indexed shows where to look.
- Export your URL list from a crawler or your sitemap and sort it. Repeated paths that differ only by a parameter or a slash are the usual suspects.
- Check which URL Google chose. If it is not the one you wanted, your signals disagree.
Which fix should you use?
Google ranks its own consolidation methods by strength. Pick the strongest one that fits the situation.
| Situation | Best fix | Strength |
|---|---|---|
| You are retiring a duplicate page | Permanent (301) redirect to the preferred URL | Strongest |
| Both URLs must stay live, for example a parameter version | rel="canonical" on the duplicate pointing to the preferred URL | Strong |
| Large site, many pages, hard to annotate | List only preferred URLs in your sitemap | Weaker hint |
The same Google page says permanent redirects are the strongest signal and that server-side redirects work fastest. It describes canonical annotations as a strong signal, and sitemap inclusion as a weaker one: pages listed in a sitemap are suggested as canonicals, but Google still decides what is a duplicate.
If you are unsure between redirect types, read 301 vs 302 redirects. For the canonical tag itself, including the self-referencing rule, see what a canonical URL is and how to set one.
What should you avoid?
- Blocking duplicates in robots.txt. Google says not to use robots.txt for canonicalization. A blocked page cannot be crawled, so Google cannot read the canonical tag on it. The distinction is explained in robots.txt vs noindex.
- Mixed signals. Do not name one canonical in the tag and a different one in the sitemap or a redirect. Google recommends consistency across methods.
- Relative canonical URLs. Use the full absolute URL.
- Linking to the duplicate internally. Link to the preferred URL everywhere on your own site.
What about near-duplicate pages you wrote yourself?
A tag cannot fix two pages that compete for the same question. If two posts answer the same query, merge them into the stronger one and redirect the weaker URL, or rewrite one to serve a distinct intent. This overlaps with keyword cannibalization, where your own pages fight each other in the results.
For syndicated content, ask the other site to include a canonical link pointing to your original, or publish a rewritten summary that links back instead of an identical copy.
A quick worked example
Say your pricing page loads at four addresses: http://example.com/pricing, https://example.com/pricing, https://www.example.com/pricing and https://example.com/pricing?ref=newsletter. A sensible setup is one preferred address, https://example.com/pricing. The first three redirect to it with a 301, since you do not need them live. The parameter version stays reachable for tracking, so it carries a canonical tag naming the preferred address. Your sitemap lists only the preferred one, and every internal link uses it. Now every signal agrees.
Does AI search care about duplicate content?
AI answers draw on pages that search systems have already crawled and understood, so a messy set of URLs makes it harder for any system to know which page is the real one. Google's page on AI features in Search says the same SEO fundamentals apply to AI Overviews and AI Mode. Clean canonicals are one of those fundamentals. Nightcrop offers a free SEO and AEO visibility audit on its homepage if you want a quick outside check.
Frequently Asked Questions
Does Google penalize duplicate content?
Google's documentation frames the issue as choosing a canonical version, not as a penalty. The practical harm is split signals and wasted crawling.
How much duplicate content is too much?
There is no published percentage. Focus on whether two URLs serve the same purpose, not on a threshold.
Should I use a canonical tag or a 301 redirect?
Use a 301 when the duplicate no longer needs to exist. Use a canonical tag when both URLs must stay accessible.
Can I fix duplicates by blocking them in robots.txt?
No. Google advises against it, because a blocked page cannot be crawled and its canonical annotation cannot be read.
Is syndicating my blog posts safe?
It can be, if the republished copy points a canonical link at your original. Without it, Google may show the other site's copy instead.