Return to site

Sitebulb Review: #1 Web Crawler Tool

The Sitebulb web crawler tool is the best alternative to Screaming Frog and a must-have tool for any international SEO audit

· Global SEO Tools

International SEO audit for Chinese companies

Performing an international SEO audit is a key element in most of my projects for my Chinese clients. Typically a Chinese company starts of their global footprint with two domains, one is and the other in most cases is .com. My clients tend to have global ambitions and aim to reach out to various international markets. However, what works well for a website targeting the Chinese mainland doesn't always work for an English website targeting an overseas audience. So my technical audit of their international websites typically show lots of areas of improvement.

The best alternative to Screaming Frog

There are tons of web crawler tools that you can use when performing technical SEO audits. Screaming Frog is probably the most famous web crawler tool, and some of you might even doubt if there are good Screaming Frog alternatives. Well, there is, and its called Sitebulb. And even though Sitebulb refers to their tool as a website crawler, to me it is more than just a crawler. As Sitebulb presents data and reports in a way that makes it easy to dig into issues and find opportunities.

Sitebulb - the alternative to Screaming Frog

Great web crawler tool

The hreflang tag is essential for global search engine optimization as it gives search engines like Google a clear signal as to which pages are for users in a specific language. And that is why hreflang for my type of projects, is on top of my website audit checklist. The way Sitebulb validates hreflang for a site is in my opinion what makes it stand out of other web crawler tools. Sitebulb probably offers the most comprehensive hreflang validation available in the market today. The Sitebulb crawler finds and checks every hreflang variation. It does so, even when you have a huge website with multiple subdomains and various sitemaps. It will pull all the data together, then give you clear recommendations to fix the hreflang configuration issues.

hreflang checker

Sitebulb is a great hreflang checker for several reasons, which I will describe below.

First of all, there are several ways to set up hreflang markup – sitemaps or HTTP headers, subfolders or ccTLDs – so you surely need to check all of them if you want the full picture of your website internationalization efforts. The way Sitebulb works it that it first lists every single alternate URL found while crawling. These get added to the crawl scheduler. And then Sitebulb crawls these themselves – no matter they were found in the HTTP header, in a sitemap, or in the HTML. This includes URLs that exist on separate domains or subdomains (e.g., a ccTLD like

Sitebulb even goes a step further and also checks the HTML lang tag, so you can see if there is a mismatch in the language declarations. Sitebulb ensures you will always have the full dataset in order to make the right recommendations.

The second reason why Sitebulb is a great hreflang checker is it validates if your hreflang tags are designed to be both reciprocal and self-referential. These are fancy words for straightforward concepts:

  • Reciprocal – if you link to a page as your Spanish translation, the Spanish page must also link back (these are often called ‘return tags’).
  • Self-referential – if a page happens to be your Spanish version, it must declare itself as the Spanish version.

If you mess up either of these, Google will ignore the hreflang annotation. Sitebulb will tell you every instance you miss either of these important tags.

Thirdly, Sitebulb validates the format of your hreflang tags. Hreflang must use a valid form to be counted. The means that countries must be declared using the ISO 3166-1 standard and languages must be declared using the ISO-639-1 standard. Claiming a land or language that does not appear in these lists would be an example of invalid hreflang (e.g. ‘en-UK’ instead of ‘en-GB’ for an English page targeted to the UK). Sitebulb checks validity amongst a series of other quality checks:

  • No broken or redirected Hreflang links
  • Use of correct syntax (e.g., mixing up the order of language and country)
  • No conflict between hreflang and rel=canonical or noindex

Last but not least, Sitebulb will check for uniqueness and consistency. When using hreflang markup, each URL may only have a single, unique association. Similarly, all other hreflang markups on other pages or sitemaps must refer to this URL as consistently the same.

With multiple sitemaps or subdomains to manage, it is easy for inconsistencies to creep in, giving the likes of Google mixed messages about your translated content.

Sitebulb will identify any duplications or inconsistent references throughout your sites and sitemaps.

Conclusion: a must-have tool for any international SEO audit

The key reason why Sitebulb for me is the best alternative to Screaming Frog, is that it addresses hreflang issues when crawling websites. Testing and validating the use of hreflang is an essential element in my international SEO audit process.

All Posts

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!