Website crawlers, more commonly known as spiders, are how search engines keep tabs on all the internet content out there. This way, when a user searches for something, the search engine can display the most relevant webpages in the search results. 

However, there are errors that can cause your website not to be crawled correctly. If this is the case, your funeral home’s website may not be showing up on search engine results pages. There are many website crawling errors that can cause this, so we’ll go over the basics below, starting with how search engines work. 

How Search Engines Work 

To see why website crawling is so important, we need to cover a little background material on how search engines work. The goal of a search engine is to deliver the best results to the user searching for information. The better the results, the more people will use the engine — and, consequently, the more revenue the engine generates. 

To determine which pages to show in response to a user’s search, the search engines maintain vast indexes of different webpages online. Webpages are added to the index via automated “spider” programs, which navigate from link to link on the web. They “crawl” the pages they find and add their content to the index. When a user enters a search into an engine, the engine filters the pages in the index according to its algorithms in order to determine which pages will best meet the user’s needs. 

Things That Can Prevent Spiders from Reading a Website Correctly 

Now, there are many things that can prevent the search engine’s spider programs from reading a website correctly. If your website can’t be read, it can’t be indexed by the search engine and can’t be displayed in the natural search results. This means visitors who should be arriving on your website won’t even see it if they use search engines to find funeral homes in your area. 

Below are a few common errors: 

Domain Name System (DNS) Error  

Woorank sums it up nicely by saying a DNS is “what allows us to navigate the internet without having to know the IP address of every website you want to visit.” If there’s a DNS error, it means the request timed out or your domain name couldn’t be found. If a search engine can’t find your website, you should contact your DNS provider. 

Server Error  

While your website was found in the DNS, there was an error loading it due to the server. This can mean several things, such as the request timed out or the connection was reset. For help fixing a server error, check out the “Server Errors” section in this Google article. 

URL Error  

These can include mis-typed URLs, deleted URLs, or URLs with limited access. To fix this, edit or update any broken links on your website using the Google Search Console’s URL Inspection Tool. 

How to Estimate Whether Your Website Can be Read by Search Engines 

There is a simple way to estimate whether your website can be read by search engines. To start, go to Internet Marketing Ninjas’ Search Engine Spider Simulator and enter the URL of one of your pages. If all the text on your page is visible within the simulator, there’s a good chance that search engines can access and parse your information correctly as well. 

Here’s an example from the homepage of Modetz Funeral Home. Although the simulator’s display isn’t pretty, you see all the text from the site’s main page was correctly read by the program. 

If you want to get more advanced with your crawl error detection, sign up for an account with Google’s Search Console and add your website to your account. Then, check the “Crawl” section periodically to identify and resolve any issues Google encountered when attempting to index your website. 

Work with a Trusted Funeral Home Website Provider 

If you work with a reputable website designer, you likely won’t run into most of these website crawling issues. Your website is a big deal, so you should work with a provider who knows the ins and outs of website design. Preferably, they’ll also have experience with funeral home websites specifically, since, as you know, they have many unique features that other websites don’t, such as memorial pages, preplanning tools, and payment platforms. 

With a Frazer-powered funeral home website, our designers have the knowledge and experience to build you a website that will get crawled correctly.