X-Robots-Tag header: noindex PDFs and files in nginx and Apache (tested)

X-Robots-Tag is the robots meta tag as an HTTP header, so it also works on PDFs, images and Word files that can't carry a meta tag. Both configs below were run and checked with curl.

1. nginx

location ~* \.(pdf|docx?)$ {
    add_header X-Robots-Tag "noindex, nofollow";
}

Result: /file.pdf and /doc.docx → X-Robots-Tag: noindex, nofollow; /index.html → no header. To noindex a whole folder instead, put the same add_header in location /private/ { … }.

The trap we hit: with both a prefix location /nf/ (using alias) and the regex location above, /nf/index.html got the header but /nf/file.pdf returned 404. nginx picks a matching regex location over a plain prefix one, so the PDF was looked up under the regex block's root instead of the alias. Use location ^~ /nf/ (stops the regex search) or nest the regex location inside the folder's block. And remember that a location with its own add_header drops headers set at server level (measured with HSTS).

2. Apache (.htaccess or config, mod_headers)

<FilesMatch "\.(pdf|docx?)$">
  Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch>

Result: identical to nginx, header on the PDF and .docx only. If it has no effect, mod_headers isn't loaded (we had to enable it in the stock httpd image) or .htaccess overrides are off.

3. Check it

curl -sI https://your-site/file.pdf | grep -i x-robots-tag

File names are case-sensitive on Linux. nginx's ~* matches .PDF too; Apache's FilesMatch is case-sensitive, so write "(?i)\.(pdf|docx?)$" if you have upper-case extensions (both from the docs; our test files were lower-case).

4. Before you rely on it

Google must be allowed to crawl the URL to see the header: a file blocked in robots.txt can stay indexed (see the robots.txt + noindex trap).
Values work like the meta tag: noindex, nofollow, none (= both), noarchive, nosnippet, unavailable_after: 2026-12-31.
Already-indexed files drop out only after Google recrawls them.

Frequently asked questions

How do I add X-Robots-Tag noindex in nginx?

add_header X-Robots-Tag "noindex, nofollow"; inside the location for those files, e.g. location ~* \.(pdf|docx?)$. Tested in nginx 1.31.6: the header appeared on PDFs and .docx only.

Why did my files 404 after adding the X-Robots-Tag location?

nginx prefers a matching regex location over a prefix location; in our test a PDF under an alias folder was served from the wrong root. Use location ^~ /folder/.

How do I set X-Robots-Tag in .htaccess?

<FilesMatch "\.(pdf|docx?)$"> Header set X-Robots-Tag "noindex, nofollow" </FilesMatch>, with mod_headers enabled.

X-Robots-Tag or meta robots?

Same directives. The header works on any file type; the meta tag only on HTML.

Written with AI assistance. Measured on 2026-10-01 with curl 8.14.1 against nginx 1.31.6 (nginx:alpine) and Apache httpd 2.4 (httpd:alpine) in throwaway containers serving an HTML page, a PDF and a .docx with the configs below. How Google treats the header is from Google's documentation, not this test. The Apify tool is ours; the APIs are free to call directly.