songtianlun/today_ai_weather
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: update robots.txt for better indexing

Browse Source 
- Add disallow rules for /admin/ to prevent indexing
- Update sitemap URL to point to the correct domain

These changes improve site indexing by ensuring that sensitive
admin pages are not accessible to search engine crawlers
and updating the sitemap to reflect the accurate URL, helping
discovery and SEO efforts.
This commit is contained in:
songtianlun 2025-01-23 19:03:00 +08:00
parent a0516f731c
commit 6544f0247c
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
public/robots.txt
View File

@ -1,5 +1,7 @@
# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
User-agent: * User-agent: *
Allow: / Allow: /
Disallow: /admin/
Disallow: /admin
Sitemap: https://your-domain.com/sitemap.xml.gz Sitemap: https://todayaiweather.com/sitemap.xml.gz