How to Fix Sitemap.xml Not Indicated In Robots.txt
How to Fix Sitemap.xml Not Indicated In Robots.txt If you see the error notice: “your sitemap is not included in your robots.txt file” during your audit, here’s how you can resolve that.
But first, let’s discuss what this issue signifies.
If you don’t understand why it’s harmful and why you should repair it, I’ll go over it fast and teach you how to fix it. We’ll also discuss the tools you’ll need to repair it and by the end of the article, I’ll show you a neat little shortcut for fixing it If you don’t have the necessary tools.
What is the robots.txt file? Think of the robots.txt file as the bouncer to your website.
It’s the same as a bouncer at a club, bar, or event venue who’s at the front door with the guest list. If you’re going to an event, you walk up and tell the bouncer your name and they search for it in their list. They either let you in or not. Perhaps they will stamp your hand or offer you a bracelet.
Maybe they’ll even offer you a VIP pass to go backstage or something. It’s a lot similar to how the robots.txt files’ function. It lets you in and provides you access to a certain portion of the website and it can also restrict access.
So, what’s the issue here?
This can be a problem because it is so fundamental. It’s the very first encounter that a bot like the Google bot or a user will have with your website, more precisely the browser, to see if they’re going to get in or not. They decide whether you’re going to be granted free access or restricted access to a website.
It’s a pretty fundamental issue so if you’re having this problem, the easiest way to fix it is with an FTP program. If you have an FTP application, you should launch it. If you do not have an FTP application, please see the link below where we review some of the most popular ones.
The one I’m using right now is known as Transmit. It is only available for Mac and while it’s not free and you have to pay for the program, it has some pretty great features that can help with the issue at hand.
If you don’t have any expertise with FTP and don’t want to learn, I understand as most FTP programs can be finicky.
If you don’t want to use FTP, stick around to the end in this post as I will introduce a shortcut for you that will help you to get around that.
If you have an FTP program open, look in your root directory for a file named robots.txt, and then open it.
It will resemble this very simple file. The first few sentences are most likely, as you’ll see.
Typically, this is what comes with WordPress by default. And then, you’re probably missing this as this sign right here reads site map, colon, space, and then the whole URL. If this isn’t here, you should insert the line exactly where you want it down below.
So copy and paste, then substitute my domain with your own. The site map XML should be the same once you’ve done that.
You can save it, and then simply exit. And there you have it, problem solved.
Now, if you don’t want to use an FTP program (I understand why you don’t as they can be a headache), there’s a nifty little workaround we can use. So, log into WordPress, go to plugins, and if you aren’t already using the Yoast SEO plugin, create a new plugin by searching for Yoast SEO.
Normally, I don’t do this since I dislike the Yoast SEO plugin. It doesn’t sit well with me and I much prefer the all-in-one SEO pack and strongly advise you to use it. However, there are a few cool tricks you can perform with it, and this is one of them.
Once you’ve installed Yoast SEO, go ahead and activate it. If you’re using another SEO plugin, such as the all-in-one SEO, you may receive a warning. Do not use two different SEO plugins.
You’re bound to run into some issues, especially if you’re running two SEO plugins but it’s fine for what we’re doing here.
So, once you’ve installed and activated it, look down near the bottom on the left-hand navigation for SEO to find the small Yoast logo, scroll over that or click on it, and then go to the tools section.
Once in the tool section, look for a feature called the file editor, which should look somewhat like this.
So go ahead and click on it. And when that opens, you’ll see your robots.txt file, and you’ll most likely be missing this line right here with your site map. cut and paste this line in, making sure to include the URL of your own site map rather than mine.
After that, go ahead and save the modifications. Afterward, return to the plugins page and then scroll down to locate the Yoast SEO plugin then deactivate it.
And now your robots.txt file has been modified, and the fact that we have just deactivated Yoast is fine. That robots.txt file will still be present.
If we go back, this is how you may find your robots.txt file: go to your domain, and then type in after your.com/robots.txt.
This will demonstrate that it’s still going to be present. So you may log in, activate Yoast, change this file, and then deactivate Yoast. This file will remain available. As a result, the file will not be deleted and the problem will be solved without the need for an FTP application.
The final step is to double-check that if you did download the Yoast plugin for this, and especially if you’re using another SEO plugin, that it’s not active by going back to your plugins page. Otherwise, you’re going to have issues.