This site is a record, not a business. BillHartzer.net publishes Bill Hartzer’s professional history and practice. It sells nothing, quotes nothing and takes no engagements — consulting and expert witness inquiries go to Hartzer Consulting.

BillHartzer.net logo mark — the professional record of Bill HartzerBillHartzer.netThe professional record of Bill Hartzer
Practice area
Engagement: AdvisoryConsultingSearch

Technical SEO

The part of SEO that decides whether a search engine can reach your pages, understand them, and keep them indexed

Abstract square grid illustration representing Technical SEO

What technical SEO actually decides

Technical SEO is the part of the discipline that decides whether a search engine can reach a page, render it, understand what it is about, and keep it in the index. Everything else is downstream of that. Content quality, links and brand strength matter enormously, but none of them matter on a URL that never gets crawled, or that gets crawled and then quietly dropped.

I have been practicing SEO since 1996, and the underlying pattern has not changed much. Most sites that underperform are not being penalized. They are being misread. A crawler follows the instructions a site gives it — internal links, robots directives, canonical tags, status codes, sitemaps — and on a large site those instructions are frequently in conflict with one another. When they conflict, the search engine decides. It usually does not decide the way you intended.

My work is to find those conflicts, rank them by how much traffic each one is actually costing, and hand back a fix list a development team can put into sprint order, rather than a ninety-page audit document nobody opens twice.

Crawling, indexation, and where crawl budget goes

Crawl budget is the practical limit on how much of a site a search engine will fetch in a given period. On a two-hundred-page brochure site it is close to irrelevant. On an ecommerce catalog with faceted navigation it is often the largest single problem, because the site is generating a near-infinite URL space out of filter combinations and the crawler spends its allocation on color-plus-size-plus-price permutations instead of on product pages.

The symptoms are consistent. Log files show heavy crawling of parameter URLs. Search Console reports large numbers of pages discovered but not indexed, or crawled but not indexed. New products take weeks to appear. Meanwhile the pages that make money are refreshed once a month.

The fixes are unglamorous and specific: constrain the URL space at the source rather than cleaning it up afterwards, keep faceted combinations out of internal links entirely, return proper 404 or 410 status codes for pages that are genuinely gone, and stop using robots.txt to solve an indexation problem. Blocking a URL in robots.txt prevents crawling, not indexing. A blocked URL that has inbound links can still surface in results with no description at all, which is the worst of both outcomes.

Duplication and canonicalization

Canonicalization is the process of telling a search engine which version of a duplicated URL is the authoritative one. It sounds like a tidying task. In practice it is where large sites leak their rankings.

The recurring failures are easy to list and easy to miss: a canonical tag pointing at a URL that itself redirects; a canonical injected by JavaScript that contradicts the one in the raw HTML; paginated series where every page canonicalizes to page one, removing pages two onward from the index along with the products only listed there; HTTP and HTTPS, www and non-www, trailing slash and non-trailing slash all answering with a 200 instead of consolidating; and session or tracking parameters minting a fresh URL for every visitor.

A canonical is a hint, not a directive. Search engines override it when the rest of the evidence contradicts it — when internal links, sitemaps and redirects all point somewhere else. That is why the fix is rarely correct the tag. The fix is to make every signal on the site agree: one internal linking target, one sitemap entry, one redirect destination, one canonical.

JavaScript rendering and structured data

JavaScript SEO reduces to one question: what does the page contain when the crawler has finished with it, as opposed to when the server first responds? Rendering is deferred and it is not guaranteed. If the primary content, the internal links, the canonical tag or the title only exist after client-side execution, you are betting indexation on a queue you do not control.

Single-page applications built without server-side rendering are the usual offenders. So are infinite-scroll category pages with no crawlable pagination behind them, and navigation built on click handlers instead of real anchor elements with href attributes. A crawler does not click. If there is no href, there is no link.

Structured data

Structured data — machine-readable markup, normally JSON-LD, that states what the thing on the page is — has become more valuable as search interfaces move toward answers rather than lists of links. It does not create rankings. What it does is remove ambiguity about identity: which organization, which product, which article, which person. Mark up what is genuinely on the page, keep it synchronized when the page changes, and never mark up content a visitor cannot see. Stale or invented markup is worse than none, because it invites a manual action and it teaches the search engine that your data cannot be trusted.

Core Web Vitals, and what they are worth

Core Web Vitals are Google's page experience measurements — largest contentful paint, interaction to next paint, cumulative layout shift. I treat them as a real but modest ranking factor and a very large conversion factor. That distinction should drive how much engineering time you spend on them.

Two things are worth knowing. First, the scores that count are field data collected from real users, not the lab score a testing tool produces on demand; a site can pass in the lab and fail in the field because real visitors are on worse connections and older phones. Second, most vitals problems were introduced for marketing reasons — tag managers loading a dozen third-party scripts, chat widgets, consent banners that shift the layout after paint, hero images served at desktop dimensions to phones.

Chasing a perfect score is a poor use of budget. Getting out of the failing range, and keeping third-party scripts under enough governance that you do not drift back into it, is worth doing.

Log file analysis

Server log files are the only record of what a search engine actually did on your site, rather than what a reporting interface summarizes afterwards. I use them to answer questions no other tool answers: which sections are crawled and at what rate, which URLs consume requests and return nothing of value, how quickly a new or changed page is picked up, whether crawling collapsed on a particular date, and whether traffic identifying itself as a search engine crawler is genuine or a scraper wearing the user agent.

Logs are also the fastest way to confirm or kill a theory. If someone believes a redirect is being followed, or that a whole section is being ignored, the log settles it in minutes. Most organizations already have this data and have never looked at it, usually because it sits with a hosting provider and nobody has asked for it.

How I work

I work as an outside technical adviser rather than as an implementation shop. In practice that means a full audit — crawl, log analysis, index coverage, rendering checks, structured data review, vitals against field data — followed by a prioritized fix list, then continuing review as the development team ships changes and as the site itself changes underneath everyone.

The prioritization is the part that earns its keep. Any auditing tool will produce hundreds of findings, and very few of them are worth an engineer's afternoon. I would rather hand over ten items that will move revenue than three hundred that will fill a spreadsheet.

Consulting engagements are handled through Hartzer Consulting.

Frequently asked questions

What does a technical SEO audit include?

A crawl of the full site, analysis of server log files, a review of index coverage and status codes, rendering checks to establish what a crawler sees after JavaScript executes, a canonicalization and duplication review, an internal linking and architecture assessment, a structured data review, and a Core Web Vitals check against field data rather than lab scores. The output is a prioritized list of fixes, ranked by likely traffic impact against implementation cost, and written so a development team can schedule it without translation.

How is technical SEO different from on-page SEO?

On-page SEO concerns what is on a page: the content, the headings, the title, the internal links, how well the page answers the query. Technical SEO concerns whether that page can be reached, rendered, understood and retained by a search engine at all — status codes, crawl paths, canonicalization, rendering, sitemaps, site speed. The two overlap and the same person often handles both, but the order matters. Technical problems make good on-page work invisible, so the technical layer is fixed first.

Does technical SEO matter for a small website?

Less than for a large one, but it is not zero. A small site rarely has a crawl budget problem. It very often has an indexation problem caused by something small and decisive: a noindex tag left over from development, a robots.txt blocking an entire directory, a canonical tag on every page pointing at the homepage, or a build where content only appears after JavaScript runs. Those are quick to find and quick to fix, and they are frequently the entire explanation for a site that ranks for nothing.

How long does technical SEO take to show results?

It depends on what was broken and on how quickly the site is recrawled. Removing an accidental noindex from an important page can show up within days. Restructuring a large catalog's URLs and internal linking so search engines crawl it efficiently is a matter of months, because the search engine has to recrawl and reprocess the site before the change is reflected anywhere. I set expectations at weeks for individual defects and a quarter or more for architectural work.

Can technical SEO fix a traffic drop after a Google update?

Sometimes, but diagnose before assuming. A great many drops attributed to an algorithm update turn out to coincide with something the site did — a release, a redesign, a plugin update, a redirect change. That is why the first step is comparing the timing against deployment history and log data rather than against a list of update dates. Where a drop really is algorithmic, technical fixes alone rarely reverse it; the response involves content quality and the site's overall standing, with technical work removing whatever compounds the problem.
Top