diff options
| author | Jan Tuomi <jan@jantuomi.fi> | 2025-12-29 23:25:02 +0200 |
|---|---|---|
| committer | Jan Tuomi <jan@jantuomi.fi> | 2025-12-29 23:25:02 +0200 |
| commit | d90cd73364708ef29baa0bf0df20cca8a07ab384 (patch) | |
| tree | 4d4958d7f4dc22b84a67c29460a76b1db5e3aac0 | |
| parent | 2ced78c03c19dd01a21e79bc5be0de55c3d3f7b1 (diff) | |
Add robots.txt trap for unruly crawlers
| -rwxr-xr-x | build-and-deploy.sh | 1 | ||||
| -rw-r--r-- | static/robots.txt | 2 |
2 files changed, 3 insertions, 0 deletions
diff --git a/build-and-deploy.sh b/build-and-deploy.sh index 4f271c7..a793f87 100755 --- a/build-and-deploy.sh +++ b/build-and-deploy.sh @@ -5,4 +5,5 @@ set -euxo pipefail ulimit -n 8192 ./update-linklog-json.sh csi -s main.scm +cp out/static/robots.txt out/ rsync -rvzP --delete --chown 80:80 out/* $RSYNC_TARGET diff --git a/static/robots.txt b/static/robots.txt new file mode 100644 index 0000000..81f3588 --- /dev/null +++ b/static/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Disallow: /wp-admin/ |