E-commerce
Specialty pharmacy · regulated retailFixing the Faceted Navigation That Was Eating a Pharmacy Catalog’s Crawl Budget
Filter combinations on a specialty pharmacy catalog were eating almost all of Googlebot's crawl budget. Getting parameter handling right took new-product indexing from an 11-day lag to under two.
- Client
- Specialty pharmacy retailer, USA
- Timeframe
- 7 months
- Scope
- Technical SEO & crawl audit · Faceted navigation indexation control · Category page intent rebuild · Core Web Vitals remediation
- +164%
- organic category-page sessions
- 99%
- Reduction in indexed parameter URLs
- 36 hrs
- New-product indexing time (from 11 days)
- $1.1M
- Incremental organic revenue, 12-mo run rate
The challenge
Dosage, form, brand-vs-generic and pack-size filters combined into hundreds of thousands of crawlable URLs, and every one of them was treated by Google as a distinct page competing for the same crawl budget as the real catalog.
Log files showed Googlebot spending the overwhelming majority of its visits on filter combinations that converted nobody, while category and product pages went days or weeks between crawls.
New formulations and restocked SKUs were taking over a week to get indexed — in a category where being first to index a newly available medication mattered directly to revenue.
Diagnostic Transformation
The Structural Shift: Before vs. After
Inspect the baseline operational bottlenecks against the deployed compounding search infrastructure.
Every filter combination crawlable and self-linking, multiplying URLs into the hundreds of thousands
340,000 parameter URLs were indexed against a real catalog of under 4,000 products. Googlebot's attention was going everywhere except where it needed to.
Critical Bottlenecks Identified:
- ✕Every filter combination crawlable and self-linking, multiplying URLs into the hundreds of thousands
- ✕No canonical or parameter handling — Google treated each facet combination as a unique page
- ✕New formulations and restocked SKUs waiting over a week to get crawled
Facet parameters brought under canonical and robots control, cutting indexed filter URLs by 99%
Parameter handling rules cut indexed filter URLs by 99% and sent almost all remaining crawl activity to category and product pages.
Engineered Infrastructure Wins:
- ✓Facet parameters brought under canonical and robots control, cutting indexed filter URLs by 99%
- ✓Crawl budget redirected almost entirely to category and product pages that convert
- ✓New formulations now indexed within two days of going live
Approach
Find out how bad the parameter bleed actually was
Ninety days of log files, not just a Screaming Frog crawl, to see what Googlebot was actually spending its visits on rather than what the sitemap claimed existed.
Give every facet combination a canonical decision
Each filter dimension — dosage, form, brand vs. generic, pack size — got an explicit rule: index, canonicalize, or noindex, instead of leaving Google to decide by default.
Rebuild category pages to carry real search intent
With crawl budget freed up, category templates were rebuilt with intent-matching copy and structure above the product grid, so the pages Google now prioritized were worth prioritizing.
What was executed
- Full log file analysis across 90 days to quantify crawl budget waste
- Parameter handling rules shipped for every facet dimension
- Canonical and noindex rules deployed for non-primary facet combinations
- Category page templates rebuilt with intent-matching copy above the fold
- Core Web Vitals remediation on the category template
- 90-day post-launch crawl monitoring to confirm indexation recovered as planned
“We didn't know we had a crawl budget problem until we saw the log files. Once that was fixed, everything else on the roadmap started working.”
Next step
Tell me what is broken. I will tell you whether I can fix it.
I take a small number of engagements at a time so each one gets real hours. If we are not a fit, I will say so in the first reply and point you somewhere better.
- Reply time
- Within one business day
- Working hours
- US Eastern & Central overlap
- Based in
- Mirpur, Dhaka, Bangladesh