How to find broken links on a website (free)

Broken links frustrate visitors and waste crawl budget. Here are two free ways to find all of them, plus how to tell real breakage from false alarms.

1. One command: wget spider (Linux, macOS with Homebrew wget)

wget --spider -r -l 5 -nd -nv -o spider.log https://example.com/
tail -n 20 spider.log

wget follows links on the same site up to 5 levels deep without saving pages. At the end of the log it prints Found N broken links. and the list. It does not say which page contains each broken link, and it does not check links to other sites (adding -H would make it crawl those sites too).

2. Python script: broken links with the page they are on

Standard library only. It crawls pages on your host, checks every link it finds (including links to other sites, which it checks but does not crawl) and prints each broken URL with the pages that link to it. Run it as python3 broken.py https://example.com/.

import sys, time, urllib.error, urllib.parse, urllib.request
from html.parser import HTMLParser

START = sys.argv[1]  # e.g. https://example.com/
HOST = urllib.parse.urlsplit(START).netloc
MAX_PAGES = 500

class Links(HTMLParser):
    def __init__(self):
        super().__init__(); self.hrefs = []
    def handle_starttag(self, tag, attrs):
        if tag == "a":
            self.hrefs += [v for k, v in attrs if k == "href" and v]

def status(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0 (link check)"})
    try:
        with urllib.request.urlopen(req, timeout=15) as r:
            return r.status, r.headers.get("Content-Type", ""), r.read() if r.status == 200 else b""
    except urllib.error.HTTPError as e:
        return e.code, "", b""
    except Exception as e:
        return type(e).__name__, "", b""

linked_from, queue, seen, broken = {}, [START], {START}, {}
while queue and len(seen) <= MAX_PAGES:
    page = queue.pop(0)
    code, ctype, body = status(page)
    if code != 200:
        broken[page] = code
        continue
    if "html" not in ctype or urllib.parse.urlsplit(page).netloc != HOST:
        continue  # external links are checked, not crawled
    parser = Links(); parser.feed(body.decode("utf-8", "replace"))
    for href in parser.hrefs:
        url = urllib.parse.urldefrag(urllib.parse.urljoin(page, href))[0]
        if not url.startswith("http"):
            continue  # mailto:, tel:, javascript:
        linked_from.setdefault(url, set()).add(page)
        if url not in seen:
            seen.add(url); queue.append(url)
    time.sleep(0.3)

for url, code in broken.items():
    print(code, url, "<- linked from", ", ".join(sorted(linked_from.get(url, ["(start)"]))))

On sitemaps.org it finished in about 20 seconds and found one dead link: the FAQ links to asg.web.cmu.edu, a host that no longer exists (URLError = the request failed before any HTTP status, here a DNS failure). Redirects are followed, so a redirected link counts as working; raise MAX_PAGES for bigger sites and keep the pause between requests.

3. What to do with each result

• 404 / 410 on your own site: fix the link on every page listed, or add a 301 redirect from the old URL to its replacement.
• 404 on another site: link to the page's new address, an archived copy, or remove the link.
• URLError / timeout: the host is down or gone. Retry later before removing the link.
• 403 or 429: often bot protection or rate limiting, not a broken page. Open the link in a browser before changing anything.

One page only

To list the links of a single page first, see how to extract all links from a web page.

Whole site, with an SEO report

Our SEO Audit Crawler & Broken Link Checker on Apify crawls your site and lists broken internal links with up to 20 pages that link to each, next to 25+ on-page checks (titles, meta descriptions, canonical, noindex, duplicates), as CSV, Excel or JSON. Billed per audited page. It checks links within your site, not links to other sites: use the script above for those.

Written with AI assistance. The wget command and the Python script were run on 2026-10-01 against a local test site with two planted broken links (both found) and against sitemaps.org (one dead external link found and confirmed with curl). The Apify tool is ours; the APIs are free to call directly.