{"slug":"indexing","title":"indexing","summary":"When the user wants to fix indexing issues from Search Console, use noindex, or implement Google Indexing API. Also use when the user mentions \"fix indexing,\" \"not indexed,\" \"Crawled - currently not indexed,\" \"discovered - currently not indexed,\" \"index coverage,\" \"noindex,\" \"noi","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-25T17:49:59.731736Z","repo":{"url":"https://github.com/kostja94/marketing-skills","stars":993,"forks":138,"license":"MIT","updatedAt":"2026-09-25T09:13:43Z"},"bodyHtml":"<hr>\n<h2>name: indexing\ndescription: When the user wants to fix indexing issues from Search Console, use noindex, or implement Google Indexing API. Also use when the user mentions \"fix indexing,\" \"not indexed,\" \"Crawled - currently not indexed,\" \"discovered - currently not indexed,\" \"index coverage,\" \"noindex,\" \"noindex tag,\" \"pages not indexed,\" \"why not indexed,\" \"request indexing,\" or \"Google Indexing API.\" For sitemap, use xml-sitemap.\nmetadata:\nversion: 1.0.1</h2>\n<h1>SEO Technical: Indexing</h1>\n<p>Guides indexing troubleshooting and fix actions. For how to find and diagnose issues in GSC, see <strong>google-search-console</strong>.</p>\n<p><strong>When invoking</strong>: On <strong>first use</strong>, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On <strong>subsequent use</strong> or when the user asks to skip, go directly to the main output.</p>\n<h2>Scope (Technical SEO)</h2>\n<ul>\n<li><strong>Fix actions</strong>: noindex, canonical, content quality, URL Inspection; verify robots.txt does not block (see <strong>robots-txt</strong>)</li>\n<li><strong>Noindex</strong>: Page-level index control; which pages to exclude and how. Complements <strong>robots-txt</strong> (path-level crawl control) and <strong>google-search-console</strong> (Coverage diagnosis)</li>\n</ul>\n<h2>Initial Assessment</h2>\n<p><strong>Project context:</strong> Read root <code>contextus.md</code> when present and load only the modules relevant to this task. Without Contextus, use available project material or user-provided facts and ask for missing information; do not create a parallel context system.</p>\n<p>Identify issue from GSC (see <strong>google-search-console</strong> for Coverage report, issue types, diagnosis workflow). Then apply fix below.</p>\n<h2>Crawled - Currently Not Indexed</h2>\n<table>\n<thead>\n<tr>\n<th>Cause</th>\n<th>Action</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Low quality, duplicate, off-topic</td>\n<td>Improve content, fix duplicates, set correct canonical</td>\n</tr>\n<tr>\n<td>Static assets (CSS/JS)</td>\n<td>See below</td>\n</tr>\n<tr>\n<td>Feed, share URLs with params</td>\n<td>Usually OK to ignore; or noindex, canonical to main URL</td>\n</tr>\n<tr>\n<td>Important content pages</td>\n<td>Use URL Inspection, verify canonical/internal links/sitemap, Request indexing</td>\n</tr>\n</tbody>\n</table>\n<h3>Static Assets (Next.js / Vercel)</h3>\n<p>Vercel adds unique <code>dpl=</code> params to static assets per deploy, creating many \"Crawled - currently not indexed\" URLs.</p>\n<table>\n<thead>\n<tr>\n<th>Do</th>\n<th>Don't</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Keep robots.txt allowing <code>/_next/</code></td>\n<td>Do not block <code>/_next/</code> (breaks CSS/JS loading). See <strong>robots-txt</strong></td>\n</tr>\n<tr>\n<td>Accept static assets in GSC as expected</td>\n<td>Do not block <code>/_next/static/css/</code> or <code>?dpl=</code></td>\n</tr>\n<tr>\n<td>Use X-Robots-Tag for static assets</td>\n<td>CSS/JS should not be indexed; no SEO impact</td>\n</tr>\n</tbody>\n</table>\n<p>Static assets in \"Crawled - currently not indexed\" is <strong>normal and expected</strong>.</p>\n<h2>Other Issue Types (from GSC Coverage)</h2>\n<table>\n<thead>\n<tr>\n<th>Issue</th>\n<th>Fix</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Excluded by «noindex» tag</td>\n<td>Remove noindex if accidental; keep if intentional</td>\n</tr>\n<tr>\n<td>Blocked by robots.txt</td>\n<td>See <strong>robots-txt</strong>; remove Disallow for important paths</td>\n</tr>\n<tr>\n<td>Redirect / 404</td>\n<td>Fix URL or add redirect</td>\n</tr>\n<tr>\n<td>Duplicate / Canonical</td>\n<td>Set correct canonical; usually OK</td>\n</tr>\n<tr>\n<td><strong>Soft-404</strong></td>\n<td>Page returns 200 but content says \"not found\" or empty—Google may treat as 404. Fix: return 404 status for truly missing pages; or add real content for 200 pages</td>\n</tr>\n</tbody>\n</table>\n<h3>Soft-404</h3>\n<p>A soft-404 occurs when a page returns HTTP 200 but the content indicates the page doesn't exist (e.g. \"Page not found\" message, empty state). Google may treat it as 404 and exclude from index.</p>\n<table>\n<thead>\n<tr>\n<th>Fix</th>\n<th>When</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Return 404</strong></td>\n<td>Page truly doesn't exist; use proper 404 status</td>\n</tr>\n<tr>\n<td><strong>Add content</strong></td>\n<td>Page is intentional (e.g. empty search results); ensure substantive content or use noindex</td>\n</tr>\n<tr>\n<td><strong>Redirect</strong></td>\n<td>If URL moved, use 301 to correct destination</td>\n</tr>\n</tbody>\n</table>\n<h2>Noindex Usage</h2>\n<ul>\n<li><strong>How</strong>: <code>metadata.robots = { index: false }</code> or <code>&lt;meta name=\"robots\" content=\"noindex\"&gt;</code> or X-Robots-Tag</li>\n<li><strong>Rationale</strong>: Not all site content should be indexed; noindex is a valid choice for many pages</li>\n<li><strong>Caution</strong>: Avoid noindex on important content pages</li>\n<li><strong>With robots.txt</strong>: robots.txt = path-level crawl control; noindex = page-level index control. Do <strong>not</strong> block noindex pages in robots.txt—crawlers must access the page to read the directive. Use both: robots for /admin/, /api/; noindex for /login/, /thank-you/, etc. See <strong>robots-txt</strong> for when to use which.</li>\n<li><strong>nofollow ≠ noindex</strong>: nofollow controls link equity only; it does <strong>not</strong> prevent indexing. To exclude from search, use noindex. See <strong>page-metadata</strong> for meta robots implementation.</li>\n</ul>\n<h3>Page Types That Typically Need Noindex</h3>\n<table>\n<thead>\n<tr>\n<th>Category</th>\n<th>Page Types</th>\n<th>Typical Meta</th>\n<th>Reason</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Auth &amp; Account</strong></td>\n<td>Login, Signup, Password reset, Account dashboard</td>\n<td>Login: <code>noindex,nofollow</code>; Signup: <code>noindex,follow</code></td>\n<td>No search value; login indexed = security risk; signup follow allows crawl of Privacy/Terms links</td>\n</tr>\n<tr>\n<td><strong>Admin &amp; Private</strong></td>\n<td>Admin, Staging, Test pages, Internal tools</td>\n<td><code>noindex,nofollow</code></td>\n<td>Not for public; avoid discovery</td>\n</tr>\n<tr>\n<td><strong>Conversion Endpoints</strong></td>\n<td>Thank-you, Confirmation, Checkout success, Download gate</td>\n<td><code>noindex,follow</code></td>\n<td>Post-conversion; no SERP value; allow link equity</td>\n</tr>\n<tr>\n<td><strong>System &amp; Utility</strong></td>\n<td>404, Internal search results, Faceted/filter URLs</td>\n<td><code>noindex,follow</code> or <code>noindex,nofollow</code></td>\n<td>Thin/duplicate; 404 = error state</td>\n</tr>\n<tr>\n<td><strong>Legal</strong></td>\n<td>Privacy, Terms, Cookie Policy (optional)</td>\n<td>Often <code>noindex,follow</code></td>\n<td>Low-value indexed; reduces clutter</td>\n</tr>\n<tr>\n<td><strong>Duplicate &amp; Thin</strong></td>\n<td>Printer-friendly, Parameter URLs, Near-duplicate</td>\n<td><code>noindex,follow</code> or canonical</td>\n<td>Duplicate content; canonical preferred when possible</td>\n</tr>\n<tr>\n<td><strong>Low-Value</strong></td>\n<td>Media kit, Feedback board (external), Thin press</td>\n<td><code>noindex</code> or index for brand queries</td>\n<td>Case-by-case</td>\n</tr>\n</tbody>\n</table>\n<p><strong>noindex,follow vs noindex,nofollow</strong>: Use <code>noindex,follow</code> for most cases—excludes from SERP but allows link equity. Use <code>noindex,nofollow</code> only for login (security), staging, or temporary test pages.</p>\n<h2>Page Removal Decision Framework</h2>\n<p>When intentionally removing a page from the web, choose the method based on whether a relevant alternative exists and whether the page should remain accessible:</p>\n<table>\n<thead>\n<tr>\n<th>Scenario</th>\n<th>Method</th>\n<th>Rationale</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Has a closely related replacement page</td>\n<td><strong>301 redirect</strong></td>\n<td>Preserves accumulated link signals and user flow</td>\n</tr>\n<tr>\n<td>Content merged into a new page</td>\n<td><strong>301 redirect</strong></td>\n<td>Direct old URL to the new canonical location</td>\n</tr>\n<tr>\n<td>Permanently deleted, no alternative</td>\n<td><strong>410 Gone</strong></td>\n<td>Explicitly signals permanent removal to search engines</td>\n</tr>\n<tr>\n<td>Deleted, uncertain if permanent</td>\n<td><strong>404 Not Found</strong></td>\n<td>Safe default; can reinstate later if needed</td>\n</tr>\n<tr>\n<td>Still accessible but should not be indexed</td>\n<td><strong>noindex</strong></td>\n<td>Page remains available to users; excluded from SERP</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Before removing</strong>: Check the URL's search traffic, backlinks, internal links, and conversion value. If the page has value, consider updating or merging rather than removing.</p>\n<p><strong>Common mistakes</strong>:</p>\n<ul>\n<li>404-ing pages that have relevant alternatives (wastes accumulated signals)</li>\n<li>Redirecting all deleted pages to the homepage (breaks user intent)</li>\n<li>Creating redirect chains (A → B → C) instead of direct redirects</li>\n<li>Removing pages without cleaning up internal links pointing to them</li>\n<li>Using <code>robots.txt</code> to block noindex pages (crawler must access the page to read the noindex directive)</li>\n</ul>\n<p><strong>Post-removal cleanup</strong>:</p>\n<ol>\n<li>Remove deleted URLs from XML sitemap; update and resubmit</li>\n<li>Update internal links to point directly to the final URL (avoid relying on redirects)</li>\n<li>For 301 redirects, ensure the target URL is in the sitemap</li>\n<li>In GSC, use URL Inspection to verify important pages; use Removals tool for temporary quick-hide (not permanent — use proper HTTP status or noindex)</li>\n</ol>\n<h2>Google Indexing API</h2>\n<table>\n<thead>\n<tr>\n<th>Type</th>\n<th>Typical use</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>JobPosting</td>\n<td>Job boards</td>\n</tr>\n<tr>\n<td>BroadcastEvent</td>\n<td>Live platforms</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Requirements</strong>: Enable Indexing API, create service account, add owner in Search Console, request quota (default 200 URLs/day).</p>\n<h2>Output Format</h2>\n<ul>\n<li><strong>Action items</strong>: Prioritized fixes</li>\n<li><strong>References</strong>: <a href=\"https://support.google.com/webmasters/answer/7440203\">Page indexing report</a></li>\n</ul>\n<h2>Related Skills</h2>\n<ul>\n<li><strong>google-search-console</strong>: Find and diagnose indexing issues in GSC</li>\n<li><strong>robots-txt</strong>: Path-level crawl control; when to use robots.txt vs noindex; do not block /_next/ or noindex pages</li>\n<li><strong>page-metadata</strong>: Meta robots implementation; noindex vs nofollow</li>\n<li><strong>xml-sitemap</strong>: Submit and maintain sitemap</li>\n<li><strong>indexnow</strong>: Faster indexing for Bing</li>\n<li><strong>canonical-tag</strong>: Resolve duplicate content</li>\n</ul>\n","files":[{"path":"SKILL.md","sizeBytes":8407,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-25T17:53:18.763953Z","sha256":"6DE33BCB327C50717C9510C60218E4D2773F9069A7849CBA1091FFA192F12B03","sizeBytes":3705},"review":null,"source":{"repositoryUrl":"https://github.com/kostja94/marketing-skills","path":"skills/seo/technical/indexing","license":"MIT","commit":"26cef3434274129040dcfc64e9c7be85ba1ada15","subtreeSha":"E7552F63CBB33249CBCE05B950C59C11613FBBC0FB9F24BE132B9897E0AE116A","lastSyncedAt":"2026-09-25T17:49:34.679591Z"},"reviewedAt":"2026-09-25T18:00:04.207801Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/kostja94/marketing-skills/tree/main/skills/seo/technical/indexing"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install kostja94-marketing-skills@llmmart"},{"target":"git","command":"git clone https://github.com/kostja94/marketing-skills.git"}]}