Screaming Frog is an SEO tool for competitive analytics

10-second breakdown:

Screaming Frog SEO Spider can crawl hundreds of websites, retrieving in-depth SEO data for link analysis, link building and keyword research. Recommended for companies looking to constantly assess their SEO performance and that of other websites.

Screaming Frog’s SEO Spider is used in small (0-50 employees), medium (51-1,000 employees) and enterprise companies (1,000+ employees).

The good:

You can crawl as many websites as you want. It’s also an excellent tool for identifying broken links and redirects. Results can be exported. With a paid license, you can save crawls that you perform and then re-upload them again to get them going once more.

The bad:

The program might crash on large websites crawls. You can prevent that by enabling it to pause on high memory usage, in the advanced settings of your configuration menu.

It’s not cloud-based, which may be an inconvenience for some.

How it works:

Let’s start by clarified an important aspect which is that Screaming Frog is a team of consultants who provide a wide range of integrated marketing services, connecting search engine marketing and SEO, PPC, content marketing, link building and social media. (For a list of tools specific to social media management, click here.)

The SEO Spider is a highly appreciated tool, free to download and to use in the lite version. With it, you can crawl up to 500 URLs at once.

If you opt for an individual license at £99/year, the 500 URL crawl limit is gone, extraction and Google Analytics integration features are available and you can access the spider’s configuration options and custom source code search. Still, you won’t be able to crawl an unlimited number of URLs with it, due to memory restrictions.

Screaming Frog SEO Spider helps with website crawling, internal link, content and meta data analysis, site mapping, general troubleshooting, keyword research and link building.

The tool crawls a subdomain that you enter. If you want to check an entire website, mark ‘crawl all subdomains’ in your configurations.

If you want to limit the crawl to a specific group of subdomains or subdirectories, you can do so using the ‘include’ or ‘exclude’ settings that can also be found in your configurations menu.

To crawl all the pages on website, you need to play with the settings a bit. The tool is set to crawl all images, JavaScript, CSS and flash files, by default. To crawl HTML only, deselect ‘check images’, ‘check CSS’, ‘check JavaScript’ and ‘check SWF’ in the configuration menu and you’ll get a list of all of the pages on your site that have internal links pointing to them. When the crawl is finished, you can filter your results by ‘HTML’, in the ‘Internal’ tab, which you can also export as .csv.

SEO Spider can also help you find broken links on your website. Again, you’ll need to de-select the types of content that you don’t want crawled such as JavaScript or CSS files, to save time and memory. After the crawl is finished, sort the results listed under the ‘internal’ tab by their ‘status code’.

Now you’ll be able to see any 404 or 302 errors. To see all the pages where you need to update your internal links, click on an individual URL from these results and then click on the ‘in links’ tab to see all the pages linked to that selected URL. You can also export this list, just go to the ‘advanced export’ menu.

If you want to check which of your pages are thin on content and could use an update, look at your internal link results and sort them my ‘word count’.

You can use Spider SEO to see all your CSS files, JavaScript files or jQuery plugins.

Checking images is also an essential part of SEO. To do that, make sure that the ‘check images’ setting is selected in your configurations, then crawl your pages. In the ‘images’ tab, you’ll be able to see and filter them by ‘missing ALT text’ or ‘ALT text over 100 characters’.

Go to the exact page where such an image is located by clicking on the ‘image info’ tab.

For further optimizations, you’ll also want to check your meta data. Screaming Frog can help with that too – as soon as your crawl is finished, to the ‘page titles’ tab, where you can filter the results by ‘over 70 characters’. You can also filter by duplicates. The same goes for the ‘meta description’ tab.

To verify your Schema markups, add “itemtype=http://schema.org” in the custom filter you’ll create in the configurations menu.

You can add up to 5 different filters per crawl. When the spider has finished crawling, select the ‘custom’ tab to see all the pages that contain your footprint. If you entered more than one custom filter, you can view results by each filter.

When it comes to keyword research, SEO Spider can help you identify which pages your competitors are linking to internally. Those will be their most popular pages that you can use in your keyword research. Once you’ve crawled a competitor’s website, sort the ‘internal’ tab results by the ‘inlinks’ column from highest to lowest, to see which pages have the most internal links. To view blog pages exclusively, deselect ‘check links outside folder’ in the configuration menu and crawl the blog folder/subdomain. Then, in the ‘external’ tab, filter your results using a search for the URL of the main domain. Again, sort the list by the ‘inlinks’ column to see which pages are linked most often.

Look at the anchor texts that your competitors are using to see which keywords they’re betting on. You can also use keywords as custom filters, like you did on the Schema markup check.

This is also helpful when developing your link building strategy. If you want to check a list of prospective partner link building websites, you can upload and crawl it in ‘list’ mode to gather more information about these pages.

After the crawl is finished, check for status codes in the ‘response Codes’ tab, and review outbound links, link types, anchor text and nofollow directives in the ‘out links’ tab. You’ll be able to see what kinds of sites those pages link to and how. You can also use a custom filter to determine whether or not those pages are linking to you already.

Screaming Frog’s SEO Spider is a reliable SEO solution. As a Google direct partner, it provides a comprehensive tool to thoroughly check your website’s SEO status, as well as those of your competitors or prospective clients.

You can count on SEO Spider for SEO analytics, duplicate content and meta data, robots.txt management, sitemap management and link building.

For complete rankings of all SEO software and tools, go here.

http://authority.org/seo-tools/screaming-frog/ http://authorityorg.tumblr.com/post/146631049227