Robots.txt for SEO – The Ultimate Guide

by Peter Mead - 18 May 2022

Robots.txt is a fundamental and important file on your website. Its purpose is to give instructions to search engine crawlers about the URLs they shouldn’t crawl on your website. It’s imperative to understand what a robots.txt file is, from a technical seo perspective, how it works, how to give instructions to the crawlers, and how to test the validity and effectiveness of those instructions.

This article will walk you through the ins and outs of the robots.txt file, so you can understand what it is and how to use it to assist your SEO efforts, and win more search results.

What is a robots.txt file?

The robots.txt file is a simple text file that sits in the root directory of your website. It gives instructions to search engine crawlers about which pages to crawl on your website. Valid instructions are based on the robots exclusion standard, which will be discussed during this article. These instructions are by way of User-Agent and Disallow.

The combination of both User-Agent and Disallow tell search engine crawlers which URLs they are prevented from crawling on your website. A robots.txt file that contains only User-Agent: * Disallow: / is perfectly valid. In this case, the instructions given to crawlers is to prevent the entire site from being crawled.

block all robots txt

(The above instructions would block your entire site from being crawled)

Crawlers access your site and add the URLs to the crawl queue. They do this for both newly discovered and previously known URLs. A crawler will first check the root directory of your website, looking for the robots.txt file. If it’s not there, they will crawl your entire site. However, if a robots.txt exists, they will crawl your website as per the directives you specify.

The main reason for updating and maintaining a robots.txt file for your website is so that your website does not become bogged down with excess crawler requests. Robots.txt is not a way to stop pages from getting indexed by Google.

A common myth is that the directives in your robots.txt file can be used to prevent pages from ending up in Google search results. The reality is that Google can still index your pages if there are other signals, such as links from other websites.

Misconfiguration of your robots.txt can have serious consequences for your website. Mistakenly telling crawlers not to access your pages can be costly. This problem can be further amplified for very large websites. You could inadvertently prevent crawlers from accessing large portions of essential pages.

Furthermore, it’s not a given that all search engine crawlers will obey the directives you have specified in your robots.txt file. Most of the legitimate crawlers will not crawl pages blocked by robots.txt. However, some malicious bots may ignore it. So do not use robots.txt to protect sensitive pages on your site.

How to use robots.txt

Search engine crawlers will check your robots.txt file before crawling the URLs on your website. If there are particular pages or sections of your site you don’t want to be crawled, pages that are not helpful to be included in search engine results pages, then robots.txt can be used to Disallow them from the crawl.

The most useful reason to include and maintain a robots.txt file is for optimising the crawl budget. Crawl budget is a term used to describe how much time and resources any search engine crawlers will spend on your site. The issue you are trying to address is when those crawlers waste crawl budget by crawling pointless or unwanted pages on your website.

Busting the Myth:- blocking indexing with Robots.txt

Robots.txt is not a reliable tool to prevent search engines from indexing pages. Pages can still be indexed in search results even if they are prevented from being crawled in robots.txt.

If it is prevented from crawling in your robots.txt file, it won’t show a detailed search results snippet describing the index page. Instead, it will give a message explaining that the description is not available because of the robots.txt directive.

indexed blocked by robots.txt

(Image credit: Barry Schwartz gives a good example of a page being blocked by robots.txt)

A page can still be indexed in a search engine if:

The page is included in the sitemap.xml.
There’s an internal link pointing to the page.
There’s an external link pointing to the website.

The most reliable ways of blocking pages from getting indexed does not include the use of the robots.txt file at all. It’s most effectively achieved by the use of the Noindex directive. When a search engine crawler reads the noindex directive, it will drop the page from the search results, effectively taking it out of the index completely.

You can effectively block a page from being indexed in one of two ways:

Use a meta tag.
Use an HTTP response header.

Block indexing with a meta tag

Most search engine crawlers will respect noindex as implemented by a meta tag. Some shady crawlers and bots may still ignore this directive, so further measures may need to be taken. However, we are mainly concerned with legitimate search engine crawlers and bots, and we are confident that they will obey this directive as specified.

Placing a noindex meta tag in the header of your page, will prevent those crawlers from indexing that page. You can block all robots from indexing by placing this code into the header of your page:

Block indexing with an HTTP response header

A more advanced way to prevent legitimate search engine crawlers from indexing your pages is to use an HTTP response header. You can configure an Apache server to return your pages’ HTTP response by placing the X-Robots-Tag in the .htaccess file of an Apache-based web server.

X-Robots-Tag in .htaccess

You will need to edit your .htaccess file on your web server. This file is read by your Apache webserver to respond to the request with an HTTP response header.

Depending on your web server setup, using X-Robots-Tag may look something like the following example:

x-robots-tag

Disclaimer: The above-mentioned methods for noindexing are advanced and can cause serious impacts to search results if misconfigured. It is recommended to seek the advice of an experienced Technical SEO before implementation.

What does a robots.txt file look like?

Not every website automatically includes a robots.txt file, so you may need to create it if you don’t already have one. You can check to see if there is one already by using the browser navigation bar and entering the URL for the robots.txt. For example, here is the robots.txt file for the StudioHawk website: https://studiohawk.com.au/robots.txt

studiohawk robots.txt

(StudioHawk’s robots.txt file)

As you can see from the StudioHawk robots.txt file, you actually may not need a lengthy and complicated file. The main aim here is to Disallow the /wp-admin/ section of the website from being crawled. This saves crawl budget.

However, we do want to Allow the /wp-admin/admin-ajax.php page to be crawled because it’s a resource that may be needed to help search engines make sense of how our website crawls and renders.

There is also a link to the website Sitemap, which assists search engines to discover it, and it shows ownership and trust. This is because only the website owner has the authority to edit the robots.txt file.

Robots exclusion standard

The robots exclusion standard is for you to give instructions to search engine crawlers. The standard provides ways to give directions to crawlers about which areas of your site they should or should not crawl. So your robots.txt should contain a list of your most important directives.

Not all crawlers or robots obey the instructions in your robots.txt file. These crawlers are often called a BadBot. These include robots looking for security vulnerabilities that may allow them to crawl or scan in sections you have specified that robots to stay out of. Common BadBots can also include spambots, malware and email harvesters.

We will focus here on the kinds of instructions we can give to search engine crawlers, and how we can help them to crawl the areas of our site we want them to. The basics of this is done by using a combination of User-Agent and Disallow.

User-Agent

User-Agent is used for specifying a search engine crawler. Depending on the specific search engine, you can specify the appropriate User-Agent you want to Disallow.

Below is a table of some common User-Agent strings to use in your robots.txt:

Search engine	User-agent
Baidu	baiduspider
Bing	bingbot
Google	Googlebot
Yahoo!	slurp
Yandex	yandex

Disallow

Disallow this tells a User-Agent not to crawl certain sections of your site. You will need to place a specifier after the word Disallow for it to be valid.

An example used to Block all crawlers:

User-agent: *

Disallow: /

Here, we are using the asterisk to specify all User-Agents, and we use a forward slash to indicate the start of all URLs.

An example used to Disallow Googlebot from crawling the /photos directory:

User-Agent: Googlebot

Disallow: /photos

Example used to more specifically Disallow Googlebot and then all other crawlers:

User-agent: Googlebot

Disallow: /keep-out-googlebot/

User-agent: *

Disallow: /keep-out-the-rest/

Non-standard robots exclusion directives

In addition to the standard directives User-Agent and Disallow, you can also make use of the non-standard directives. Please keep in mind that there’s no guarantee all search engine crawlers will follow all these non-standard directives. However, for the main search engines, these are fairly consistent.

Allow

Allow is supported by the major search engine crawlers, and it can be helpful when used in conjunction with a Disallow directive. Use this directive to Allow crawlers to access a file within a directive that may have a Disallow on the directory.

We need a little specific syntax to ensure that the major search engine crawlers will respect the Allow directive. Make sure you place your Allow directives in the line above the Disallow.

An example used to Allow a file inside a directory with a Disallow:

Allow: /directory/somefile.html

Disallow: /directory/

Crawl-delay

Crawl-delay is not supported by all the major search engine crawlers. It’s used to limit the speed of the crawler. This is commonly used when your website is experiencing performance issues due to too much crawler activity. However, poor site performance is generally due to inadequate web hosting and should be addressed by improving your hosting.

However, Google does not respect Crawl-rate, and will simply ignore this directive if you have it specified in your robots.txt file. If you want to rate-limit Googlebot crawler, you need to go to the Google Search Console Old version and adjust it there.

An example used to set Bingbot Crawl-delay:

User-agent: Bingbot

Allow: /

Crawl-delay: 10

Sitemap

It’s considered best practice to include your XML sitemap in your robots.txt. It’s thought that this can aid in the discovery of your sitemap, and therefore help files be discovered and crawled faster.

Example of your XML Sitemap in your robots.txt file:

Sitemap: https://www.examplesite.com/sitemap.xml

Wildcards

Wildcards are supported by all major search engine crawlers when implemented correctly. You can use these wildcards to group files together by file type. The wildcard replaces the name of the file and will match any filename where the wildcard is used.

Example to prevent crawling of all .png images and .jpg images in a specific directory:

Disallow: /*.png

Disallow: /private-jpg-images/*.jpg

Adding and editing robots.txt on your server

You will need to have access to be able to add a robots.txt file. Some CMS platforms, such as WordPress, allow you to add/edit your robots.txt.

Add a robots.txt file

If you have access to your web hosting files via a cPanel or some other similar hosting management console, you can create a robots.txt if you don’t already have one.

You may need to add a robots.txt by uploading it via FTP. This is a commonly performed task, and your web hosting provider can point you in the right direction for this.

Edit a robots.txt file

If you have the Yoast plugin installed for WordPress, you can edit your robots.txt file directly from within the dashboard.

By default, the robots.txt file for WordPress will look like the following.

standard wordpress robots.txt

To begin editing your robots.txt with Yoast, follow these steps:

Click on ‘SEO’ in the left-hand side Yoast menu in the WordPress dashboard.
Click on ‘Tools’ from the expanded settings options.
Click on ‘File Editor’, and you can now edit your robots.txt.
Note: Please make sure file editing is enabled, otherwise this option will not be available.
Make the desired changes to your robots.txt.
Save the file to make sure the changes take effect.

If you want more detailed instructions, they are available on the Yoast website.

If you cannot access the robots.txt via your CMS, you will need to access your web hosting files directly via cPanel or your web hosting console.

Testing your robots.txt

Testing any changes before setting them live is a critical activity for SEOs to respect. A mistake in your robots.txt has the potential to stop crawlers from accessing your entire website. This can lead to detrimental outcomes for search results and lead to a severe drop in rankings and heavy traffic losses.

We recommend that you test and validate your robots.txt before making them live. Thankfully we have a handy tool for testing robots.txt by Google. It’s available from within the Old version menu in Google Search Console.

Key takeaway

Robots.txt is a vital component of your SEO efforts. If it is configured poorly or misconfigured, your SEO efforts could experience real impacts. You will want to make sure you have not mistakenly blocked crawlers from the most important parts of your website.

You will want to make sure to block crawlers from excessive crawling of unimportant URLs. Blocking crawlers from accessing unimportant URLs will save crawl budget. You will want your crawl budget spent on the pages that are important to search engines.

Noindexing pages is an incorrect use of robots.txt. Keeping pages out of the search results is best achieved by using meta robots noindex, or X-Robots-Tag. Otherwise, a page may still end up being indexed if it is referenced by other pages.

Mastering robots.txt is a very powerful yet fundamental skill for every SEO. Understanding how it works and when to use it will help you take more control over your SEO results.

JavaScript SEO Devs and SEOs getting on the same page

by Peter Mead - 20 Apr 2022

Both SEOs and Devs today have a lot to deal with regarding the issues faced when getting JavaScript content indexed by Google. JavaScript websites are becoming a lot more commonplace. It’s certainly no big SEO secret that many of these sites are having issues.

Organic website traffic is a hard-fought and won battle, and sometimes SEOs and Devs have trouble seeing each other’s perspectives. It’s really worth getting to understand each other’s objectives and getting on the same pages so you can solve the many issues which may arise with JavaScript SEO.

Understanding JavaScript SEO

Optimising for JavaScript websites

JavaScript SEO deals with the Technical SEO elements of websites built with JavaScript, particularly how you can make it easier for search engines to crawl, render and index your website.
What is javascript seo
(img: Moz featured snippet)

Both SEOs and Devs need to focus on ways to allow search engines to crawl, render, and index your JavaScript site. That’s essentially what JavaScript SEO is. You need to make it easy for Google to index your site effectively.

As JavaScript websites become ever more more popular, it’s essential that you take the SEO journey to get on the same page with developers. Your website may have been developed with a certain functionality that required JavaScript to achieve it. However, it’s possible the SEO implications may not have been considered, resulting in poor organic search results.

JavaScript Frameworks, Vue, Angular, React

Devs working with JavaScript will already know that it’s a popular web scripting language. Used in conjunction with HTML and CSS, JavaScript allows web pages to have dynamic functionality, which standard HTML pages would not allow for. This functionality includes all kinds of animations and interactivity features, which can only be accomplished with JavaScript.

To achieve this dynamic functionality, many developers are electing to build websites with JavaScript frameworks alongside or instead of using a well-recognised CMS.

Popular JavaScript Frameworks

Some of the more popular JavaScript frameworks include:

Popular javascript frameworks

Image credit link: Front-end frameworks popularity of React, Vue, and Angular

Although it’s quite doable to include various dynamic functionality within the HTML of a page, the rise of single-page and multiple-page web applications has seen developers opting to build complete websites with JavaScript frameworks.

Using a JavaScript framework to build an entire website can often mean less success with Google. This approach can cause unforeseen issues with crawling, rendering, and indexing, often resulting in your site losing out on Google love.

Why you need JavaScript SEO

Most content-driven websites don’t have complex functionality. They are mainly for the purpose of acquiring customers or sales. The goal for moving to these kinds of complex JavaScript websites would be in the realms of the developers’ preferences and experience.

Suffice it to say, unless these kinds of websites are properly tested for common crawling, rendering, and indexing issues, then you may not have much success with the search engines. You’ll need to get on the same page to understand JavaScript SEO and how to optimise for it, whether you’re the developer who built the website or the SEO who’s trying to get it found with Google.

You will need to take all the necessary steps to ensure your website is SEO friendly, allowing Google to crawl, render, and index in order to get higher positions in the search results.

How Googlebot processes JavaScript Websites

The three phases of JavaScript processing

Google now renders JavaScript content much better than it did several years ago. It now happens near instantaneously, whereas in the past, it could have taken weeks.

However, just because Google has improved its JavaScript rendering, doesn’t mean that your pages will automatically get indexed if there are issues. To understand the kind of issues you could face, Devs and SEOs could schedule a meeting to walk through how Google does this.

Google processes JavaScript in three main phases:

Crawling
Rendering
Indexing

The diagram below may help you get an understanding of the process. There’s an extra step at the end added on for ranking, which can only come after the whole process has been completed successfully.

crawl render index process

Image credit link: more about this on Google’s documentation

Crawling, discovering pages and resources

SEOs and Devs can meet and discuss how crawling is all about the content discovery phase. This is where Google tries to discover all your pages and resources. Since the advent of mobile-first indexing, Google will likely be crawling with Googlebot smartphone crawler.

Crawler sends GET requests to your web server.
Your web server gives response headers, & file contents
This content is saved to the crawl queue for processing

Crawl requests usually come from Mountain View, CA, USA, but Googlebot can also crawl locally in your local area outside of the USA. For this reason, it’s important to pay attention to any geo-based page redirects which may prevent Googlebot from accessing essential areas of your site.

Google wants to crawl and store ALL of the resources to build the page, including HTML, JavaScript files, CSS, and any APIs. Then all this content goes into and begins to build the crawl queue, with its own specific crawl budget emerging.

Processing, preparing HTML & Resources for rendering

During the project, you can discuss how Google looks for resources (CSS, JS, Images, etc.) and links to pages to build the page later and provide relevance and context.

Finds resources and links to pages and files needed to build the page.
These are added to the crawl queue and cached.
Prioritising and scheduling is based on the crawl queue.

Google will look for valid anchor links tags with a href attribute. If there are JavaScript links to resources and pages, these may not get crawled or processed.

At this time, the most restrictive robots directives are chosen from either the HTML or Javascript versions of the page as Google encounters them. For example, a noindex directive overrides an index, and a noindex in HTML skips the rendering.

Rendering, seeing Pages the way users see them

Get an understanding of how the processed pages are placed in the render queue:

Renders the changes made to the Document Object Model (DOM) by Javascript
Renders Using “evergreen” headless Chrome

As you can imagine, rendering trillions of pages on the web is massively resource-intensive, so Google needs to have shortcuts. So it renders all the resources that were cached during the processing stage.

However, this may cause unexpected results if the state of any shared resources has changed since it was cached. Take, for example, a timer-based popover stored in cache for rendering. If more time has elapsed and the popover appears on the screen, then Google may render this, which may not be the desired page for Google to index.

It’s tempting to use the 5-second rule for the rendering timeout. In reality there’s actually no fixed 5-second timeout; rather, Google uses indicators such as inactivity or blocked processing. Google has said that if it renders in Mobile-Friendly Test, then it should get indexed. Of course getting indexed is not always a simple process.

Indexing and crawling content in two waves

It’s also crucial for both SEOs and Devs to understand that Google goes about this whole process in what it calls two waves. During the first wave, any of the static HTML is to be indexed. Then on the second wave, any of the additional JavaScript rendered content gets indexed.

Crawling indexing two waves

If you’re developing JavaScript websites, it will help you to know that in May 2019, Googlebot was upgraded to the latest version of the Chromium headless browser. It’s now called “evergreen,” making it fully compatible with the up-to-date versions of JavaScript. But don’t be fooled because there still can be indexing issues, which have a lot to do with crawl budget.

Even though the time from crawl-to-render is faster now than ever, there are still unforeseen issues your JavaScript page may face. These issues mostly depend on whether Google can actually render important content created with JavaScript.

The pages built with JavaScript have to go through the extra rendering step, which delays things slightly to save crawl budget. Crawl budget costs Google processing time and therefore is expensive, so being added to the Web Rending Service queue is for efficiency’s sake.

Developers should note that the render queue is held in a cache, so it’s not real-time. Google will render the page at a later stage based on the resources it crawled and stored in state. This is regardless of whether the state has now changed due to a timeout elapsing or a user interaction with the page.

Common Issues preventing Rendering

Even though your page is queued for rendering, it may not actually render as expected. A range of Dev related factors can be preventing Google from rendering your page properly, such as:

Robots directives blocking important .js resources, preventing Google from crawling and therefore rendering the content.
Timeouts causing network resources to take a long time to become reachable, so Google gives up and does not render.
Errors within the execution of the code resultant from state or network connectivity issues, such that JavaScript is unable to return HTTP status code errors.
Lazy loading images and text or other content, which is integral for the content, however, is not yet available in the cached render queue.
Hyperlinks generated by JavaScript are not crawlable, because Google does not click links, so use static links for them to be rendered.

If any nuance of these things goes wrong as Google renders the content it has in cache, then the page Google sees, and indexes may be quite different from the page developers are expecting to be rendered.

JavaScript Rendering Methods

As already stated, issues you may encounter with getting Google to index your JavaScript content will mostly be determined by the level of successful page rendering. However, the very best outcomes will be obtained when SEOs and Devs work together towards the common goal of attaining a clean and efficient rendering of JavaScript pages.

Therefore the task of achieving that wonderful clean SEO-friendly render becomes a shared project between SEOs and Devs, and there’s a lot each party can bring to the table. An ongoing process of best practice and trial and error will most likely need to be embarked upon.

You may think that because rendering is so important that what you see is what you get, and that Google renders your page just the same way as a visitor to your page might. But there’s actually a wide variety of rendering methods a developer can choose to implement, depending on how the developer has gone about approaching things.

Common Rendering Methods

Some of the most common JavaScript rendering methods include the following:

Pre-rendering
Client-side rendering
Server-side rendering
Dynamic rendering
Hybrid rendering
Isomorphic JavaScript

This article from Google covers a variety of rendering methods

For the purpose of this article, we will focus on three main rendering methods – Client-Side Rendering, Server-Side Rendering, and Dynamic Rendering .

Consider the costs of DOM vs. vDOM

Developers will know the DOM provides a tree structure of web elements within the HTML document. Devs take advantage of the JavaScript framework features to manipulate the web elements on the DOM. I will avoid diving too deep into the nuances in this article, but if you want to dive deeper, here is documentation about the DOM on Google.

It’s important that SEOs and Developers understand how the different frameworks use the DOM. Vue and React use a Virtual Document Object Model (vDOM), while Angular uses a real DOM. Individual elements can be updated with a vDOM instead of having to update the entire tree, which is the case with a real DOM.

PageSpeed has long been a major concern, and more now than ever, as Core Web Vitals become an integral consideration for Web development and SEO alike. From a performance perspective, vDOM is considered faster than a real DOM, and pages built with Vue and React are often faster than Angular.

It’s helpful for the SEOs and Devs to work together, looking at methods used for adding elements to the DOM and how they are also being rendered as a result.

Client-Side Rendering (CSR)

Open and constructive conversations around the impact of Client-Side Rendering (CSR) can be eye-opening for SEOs and Devs. With this method, the Googlebot renders the JavaScript content using the DOM as it’s fetched from the render queue. You are aiming to render the page content relying on the browser’s capability rather than loading content from HTML.

This is where most SEOs encounter issues as you start coming up against various unforeseen discrepancies between what Devs are rendering to the browser compared to what Googlebot can render successfully, or not render properly at all.

A worst-case scenario is what developers should do when Googlebot sees a somewhat empty HTML document when rendered. Begin troubleshooting with some of these tools:

Server-Side Rendering (SSR)

Discussions between Devs and SEOs about Server-Side Rendering will involve acronyms such as SSR and will likely be the most complex for developers to achieve. With this method, it’s the server, not the client, that renders the JavaScript and passes the rendered page to the client.

Google still crawls the page, but since rendering is already done on the server, it goes straight to the indexing stage. As JavaScript was used to render the page, it’s now treated like any other HTML page, which means JavaScript-related rendering issues are avoided.

If you decide to work with the SSR method, you may consider rendering tools such as:

Dynamic Rendering

As you progress your discussions, it might become favourable for both Devs and SEOs to opt for a dynamic rendering methodology. Here we are offering Google a static version of the page while giving the dynamically rendered version to other user clients.

This is a hybrid solution serving the content based on the user agent that requested it. A best of both worlds approach, delivering static HTML for the bots and crawlers, while delivering the JavaScript version for real-world users when identified as such.

The bots receive a version of the page designed to be machine-readable. It contains a paired back version of the content, with text and links so it can be more easily parsed by crawlers.

The human users receive the JavaScript version of the page. They experience all the fully interactive features which you have designed and developed according to your user engagement strategy.

dynamic rendering model

Image Credit: Google

Colluding your prerendering tests

Pre-rendering using a hybrid or dynamic rendering approach may be a little tricky to get your head around at first as Dev or an SEO because you may not understand how to test what Googlebot crawler is seeing.

One way to get the conversation going between Devs and SEOs is to firstly look at using a reliable solution such as prerender.io (https://prerender.io/). You could also use Puppeteer, or Rendertron.

To test how the rendered page is being interpreted by Google you can run the page through the Mobile-Friendly Test tool. This tool uses Googlebot smartphone crawler to crawl the page. It sees the prerendered version that has been set.

Another good way to test prerendering at scale is to use Screaming Frog SEO Spider (https://www.screamingfrog.co.uk/seo-spider/) to perform a crawl across your site, or sections of your site.

Configure Screaming Frog User-Agent, to use a Googlebot crawler so that the rerender triggers the crawler to look at the prerendered version of the page.

user-agent selection screaming frog

Also, make sure you have your spider configured, so it does not have Rendering with JavaScript enabled. The spider can look at the prerendered version you have selected with your dynamic rendering method.

Javascript rendering setting screaming frog

Isn’t this cloaking?

SEOs and Devs may at first be a little concerned about switch content served to Google detecting the User-Agent. Any SEO who has been around for while will be familiar with the idea of cloaking, which uses similar methods to serve different content to Google than the user Sees.

It has been declared that the practice of cloaking is against Google’s policies, and may result in penalties if it’s implemented on your website. However, Dynamic rendering methods are permitted and recommended by Google, as long as your rerendered version of the page does not show different content to the version being served to the clients.

It’s an extremely low-risk practice; however, to avoid any unforeseen issues, Developers and SEOs can read the Google documentation explaining how dynamic rendering can be set up.

Your Collaboration Journey

At the end of the day, Devs and SEOs can agree on the potential for JavaScript to cause SEO-related issues for crawling and indexing your website’s content. Beginning a collaborative JavaScript SEO approach can really be about finding ways for the Devs and SEOs to get on the same page.

Taking the time to understand the issues from both sides can help us get the SEO success we are hunting for. By working together for the same intended outcome, we can learn a lot more about the kinds of JavaScript issues faced together, and then make better-informed decisions about the kinds of rendering approach that are best for us.

What Is SEO and How Does SEO Work?

by Lawrence Hitches - 2 Apr 2022

Of all the online activities, 68% of these begin with a search engine. When you look at the people doing the searching, 75% of these searchers will not navigate past the first page of results.

The world of SEO (Serch engine optimization) is vast; it contains terminology we need to learn. Tactics to turn into actions and solid strategies that need forming.

But before we can do all those things, we need to understand the fundamental question, “What is SEO”?

You need to know that SEO is a potent tool that you can use to make your website more visible online and increase your website’s traffic, much like getting more people through a shop’s door.

So, please keep reading for the complete guide on what SEO is and how it works!

What Is SEO, And How Does SEO Work?

SEO is the abbreviation for search engine optimization.

Search engine optimization (SEO), in its simplest form, refers to the practice of increasing your website’s visibility in Google and other search engines when customers are looking for your company’s products or services. The more prominent your website is in search results to attract new and existing customers, the more likely it is.

SEO is how you make this happen; it’s the strategy and actions one takes to get their business website from page 54 to page 1 on search engine results.

How Does SEO Work?

When a search request is used, the search engine needs to decide which page to show according to what is being asked. These search engines, like Google, use algorithms or a whole list of rules to make this happen. These algorithms have become exceedingly complicated, taking into account hundreds, if not thousands, of distinct ranking parameters to establish rankings.

Search results display in an ordered list, and the higher a site can rank on that list, the more visitors it will receive. Usually, the results at the top of a page for a typical search query will receive 40-60% of the overall traffic for that specific search term, but the results that follow that will receive much less traffic.

Significantly few searchers will go past the first page of search results. So even slight increases in your website’s ranking can turn into increased visitors and, perhaps, it could even generate leads and grow a business’ bottom line.

The competition in SEO is fierce, and to stand out, you’ve got your work cut out for you.

The Three Pillars of SEO

It is a fact that the triangle is the strongest, sturdiest shape. So it’s only fitting that the pillars upon which SEO stands three; these three pillars cannot operate in isolation, much like the three points in a triangle.

These three pillars are:

Technical SEO
Content
Backlinks

Let’s have a deeper look into what makes up these three pillars and how they matter to each other.

What Is Technical SEO?

These elements drive your site’s performance and crawlability from behind the scenes. Your site should have a clear and straightforward structure for search engines to scan and index your content. The site’s URL structures should be logical.

You may need to arrange things differently depending on the sort of site, but in general, you’ll want to break your site down into relevant subjects or chunks and make it easy to browse between topics and sections.

Technical SEO is where you focus on the speed of your website, how mobile-friendly your website is, and navigation and the user experience from your technical SEO tactics.

What Is Content? What is Search Intent?

Content SEO is exactly that, all the elements of SEO on every page of your website. These are things like keyword usage and placement throughout the pages on your website.

The tags on your website should be relevant to your other content. Things like your descriptions and titles need to be carefully constructed.

Lastly, images and videos are also helpful to increase your search engine ranking. Make sure they’re also relevant to the content on your website and have valid alt text to help with searching.

What Are Backlinks?

Backlinks or off-page SEO is the connection between your website and others. When a website links to yours, that’s a backlink; it’s an endorsement of trust from another entity. Google can understand the connection between this as a “vote of confidence” that a website is credible. In SEO, gaining these backlinks is called link building has to be done strategically and with quality in mind.

Differences Between Paid and Organic Search

The difference between organic and sponsored search is straightforward: the price.

Sponsored or paid search focuses on paid ranks and options such as the name suggests, whereas organic search focuses on unpaid rankings in search results. Companies employ SEO to improve their site’s visibility or rankings in organic search results. On the other hand, Paid search allows users to pay for a prominent position in search results.

Similarities Between Paid and Organic Search

There are many parallels between paid and organic search. It’s crucial to be able to distinguish between the two.

Paid search results are typically seen at the top of the page. They’ll have a tiny note on them that says “ad” or something similar. You can readily tell what an ad is since even if we don’t always think of digital marketing as advertising, it is essentially advertising.

Paid search results will always have a tiny ad indicator directly beneath the clickable headline, but paid search and organic search are becoming increasingly similar.

The following are the search results:

Have headlines that can be clicked
Have body copy that isn’t clickable
Have clickable site links (the tiny links beneath the main ad space) so that there are a variety of options

Realistically, it takes three, six, or twelve months for search rankings to move, and most people aren’t willing to wait that long. Paid search assists you in attracting traffic to your website right now while also expediting optimizing your landing pages for more conversions, which pays off in the long run.

Why Do Search Engines Care About SEO?

Imagine a world where we didn’t have any entities organizing the world of search. If you were specifically looking for information on a particular breed of dog, imagine the sheer and immeasurable amount of irrelevant information you would have to sift through to find the information you want.

You would be completely overwhelmed with information about dog breeds you don’t care about, dog and pet products that you don’t need or want, dog breeders, dog handlers, dog trainers, vets, and much more.

Every search engine cares about SEO because it works for the website looking for an audience and the searcher.

Search engines are the internet’s way of saying, “I will match this information with this searcher because they are a match as a result of a long list of variables in the backend.”

How Do Search Engines Work?

When you’re starting to work out how search engines work, you’ll find buzzwords like crawling, indexing, and ranking. Sifting through the anagrams and the tech words can be intimidating, but understanding the process behind search engines doesn’t have to be confusing.

Spider bots crawl the web, which is how search engines work. These web crawlers effectively track links from one website to the next to identify fresh content in the search index. Relevant results are taken from the index and ranked using an algorithm when using a search engine.

It’s understandable if that sounds complex. However, if you want to increase traffic to your website by ranking higher in search engines, you’ll need a rudimentary understanding of how search engines detect, index, and rank content.

What Is Crawling?

Crawling is the first step. Search engines send web crawlers to find new pages and gather data about them. These web crawlers are also known as ‘spiders’ or ‘robots.’ Their goal is to find new web pages and examine the content of pages they’ve already visited to see whether it’s changed or updated.

Search engines crawl web pages by following previously found links. When a search engine scans your homepage, it will look for another link to track and may follow the link to your new blog post if you have a blog post connected to your homepage. Search engines are occasionally told not to crawl certain web pages, so they are left alone.

What Is Indexing?

Indexing is the next stage. When a search engine determines whether or not to use the content it has crawled, it is called indexing. A search engine will add a crawling web page to its index if judged worthy. This index is used in the final stage of ranking.

When a web page or piece of material is indexed, it is filed and saved in a database from which it can be retrieved at a later time. The index contains most web pages that have unique and valuable material.

What Is Ranking?

The third and most critical phase, ranking, is the most important. Search engines sort or rank the results for any given keyword to provide the searcher’s most valuable and relevant results. After the crawling and indexing stages are completed, ranking can begin. Your site can be ranked once crawled and indexed by a search engine.

It is said that search engines use over 200 ranking signals to categorize and rank material, all of which fall under the three pillars of SEO: technical optimization, on-page optimization, and off-page optimization. The existence of a keyword in the title tag, the loading speed of the web page, and the presence of a website plus its reputation are all examples of signals that search engines use to rank web pages.

How To Make SEO Work For Your Business

With the countless ways you’re told to find success with SEO, it can be tricky to figure out how to do things yourself. But, making SEO work for your business is a combination of tactics working together to create a well-oiled, optimized machine.

Focus On The Right Keywords

Years ago, one would’ve found a bunch of keywords that could fall under the relevant industry, and they would’ve stuffed their website full of them. This would’ve worked back then.

Nowadays, there is far more strategy behind finding the right keywords for your brand, your business, and your website. The keywords should appear in the URL name, strategically throughout your website, and within your content.

Focus On The Visitor

Search engines love it when a website prioritizes its visitor experience over keyword stuffing and other SEO checkboxes.

Have a simple, easy-to-use website. Make it apparent to visitors who you are and what you have to offer right away.

Use many headings and include your keywords in the headers to boost readability. For search engines, titles stand out and will help you rank better.

The speed of your website is hugely significant for SEO. The last thing you want to do is irritate your visitors by having a slow-loading website.

Finally, check to see if your website is mobile-friendly. Websites that are difficult to view on a smartphone are now penalized by search engines; your website should be design-responsive depending on what device it is being considered.

Content Must Be High-quality

High-quality content researched, written, created, and published with the relevant audience in mind will increase your search engine rankings with minimal effort.

With this kind of content, you’ll likely be giving the information that your audience is seeking, which will increase your website’s traffic. Again, search engines will begin to rank your content as authoritative and decide it should be shown to more people.

If you’re considering getting assistance, know that some professionals can do both content and SEO.

Link Building Should Be a Priority

The thought process behind link building is simple:

The more external websites that point to your website, the better it is for you. But, you’ll be looking for quality websites to do the pointing. The better related these external websites are to your industry and field, the more sense your link-building strategy will be.

The quality of links needs to outweigh the number of connections to make SEO work for your website and business.

Setting SEO Objectives

Setting SEO objectives is easier said than done. How does one know what they should be setting out to achieve? What are SEO goals realistic and achievable? What kind of timeframe does one look to accomplish SEO objectives within?

There is no one-size-fits-all approach for SEO objectives. Remember, Rome wasn’t built in a day, and neither can your SEO show results overnight.

The Most Obvious SEO Goal

When it comes to setting SEO goals, there is one that outranks the rest, and it is to get more website traffic. You can achieve this by being productive and sticking to your goals.

Once a brand receives more website traffic, the positive circle begins. Search engines will start to rank the website higher, and hopefully, more website traffic means more quality leads for your business.

And at the end of the day, more leads generated should ultimately mean a better bottom line. But what about SEO objectives that can help you get to this point?

Examples of SEO Objectives

To achieve the ultimate SEO goal of increased website traffic, a better search engine ranking, and an increase in sales, one has to detail their SEO objectives.

Setting goals like these SEO objective examples will likely get you closer to your page one position.

Get Visitors To Stay For Longer

It’s damaging to your website’s ranking if people visit your website but don’t stay and digest your content.

A good SEO objective is to look into ways to increase the time that visitors stay on your website.

Optimize Page Speeds

You have split seconds to make an impression on your visitors, so pages that take too long to load will likely drive away your audience members.

Another excellent example of an SEO objective is to prioritize the loading speed of your website. To dig into each page and get it working at its absolute best in terms of loading is a good SEO goal.

Improve Links

One of the SEO objectives you should tackle is improving your website’s internal links. So, your goal should be to analyze every link that appears and remove broken ones and improve on those that are no longer relevant to your business or brand.

Setting Objectives For Different Types of Businesses

Generally speaking, different businesses will likely have different goals regarding their online objectives. Some brands will be after online awareness growth, while others want to generate quality leads.

The key to this is understanding your business’s critical objectives outside of digital and then adding to those goals to help you achieve them with your digital and SEO tactics.

For example, an eCommerce website will employ entirely different SEO tactics than an international school. Why? The eCommerce store is looking to make a sale right then and there on their website, whereas the school is looking for people to make an inquiry or fill in the contact form.

This means that each business needs to work on its SEO to complement what its type of business is trying to achieve.

SEO will most likely work towards achieving these things:

Driving organic traffic to your website
Performing lead acquisition and marketing directly
Reputation management
Influencing their industry and appearing in news
Driving brand awareness
Increasing eCommerce sales

Choosing one of the above for your business is based on your type of business. Once you’ve chosen your objective and goal, you can work on the tactics that will help you achieve it. Ensure that you have the means with which to measure your performance before you start any campaign or make any alterations or changes.

What’s incredibly important is knowing where you started and how far you have come with your SEO. Otherwise, how will you know if anything you are doing is working? Make sure that you record the foundation you started on and set up regular intervals to check your performance.

Successful SEO Campaigns

The fundamental answer to “What is SEO?” has been given. It would help if you had an in-depth understanding of search engine optimization and what your business wants to achieve. Getting there, however, may not be so easy.

Remember that SEO is a journey and not a destination. Your SEO efforts will constantly adapt and mold to fit the ever-changing digital world. You will never be done, so set aside the time to measure your performance and tweak the parts of your SEO strategy that need it.

The most crucial aspect of your SEO journey is measuring your progress and measure your performance, which is based on your objectives and your goals.

Do you feel like you still need help? Book a call with me, and we’ll get you all the SEO help you need.

Brand Tone of Voice – What Is It and Why Is It Important?

by Matthew Farmer - 21 Jan 2022

There are a lot of brands out there in the digital landscape. It can be tough to stand out from all the noise. So many businesses are trying to sell their wares. So how can you get your audience to listen to you? How can they hear your voice amongst the white noise?

What is Brand Tone of Voice?

A brand tone of voice is an excellent step in distinguishing yourself from your competition. How do you determine what your tone of voice is? How do you know if it’s the right tone of voice for you? And why does it matter? If you have the best products or services in the world, why should you even bother with a tone of voice?

A brand tone of voice is a distinct personality for your brand. Think of the cheeky M&M’s or the lifestyle tone of voice that Coca-Cola offers you. You want to think of your target audience and how to speak to them, and you also want to consider how your content speaks in general. If you are your brand, that consideration is vital.

It is the Values of your brand

You need values for your company, things that you believe, and ways in which your business operates.

StudioHawk has Four main values we operate by:

Embrace Opportunity – if we see a chance to do something good, for a client, for the community, or the business, we’ll take it. At the start of 2021, we took a chance and started a separate Content Writing team, and it exploded. We also took a chance to launch a distinct Digital PR team, and it too has flourished.
Integrity– there are many dodgy people in the digital space, and we want to stand apart from them and be proud of who we are and what we do. We do not abide by lock-in contracts. We believe we’re good enough to keep you as a client no matter what. On the flip side of that, we’ll be honest with you and say if you need us or not. We’re not going to sign you up just to take your money.
Transparent – we are open and honest with what we do and how we do it. We’re not hiding any SEO secrets. If you want to know more about SEO, check out our Hawk Academy and learn some basics for free.
Fun – you have to enjoy your work and your workplace; otherwise, it just becomes a grind.

What you read on our blogs reflects these values. We try to inject some fun, we tell you how things are, honestly, and we stand behind what we say. If we wrote any differently, it wouldn’t sit well with us, and you’d notice we weren’t ourselves.

Look at your business and the values you uphold. How can you reflect those values in the words you write?

It is the Personality of your Brand

What kind of personality does your brand have?

Is your brand like Captain America?

Loyal
Honest
Upfront
Will not cuss.

Is your brand’s personality more like Tony Stark?

Brash
Confident
Doesn’t work well with authority?
Smart and a bit of a smart arse?

It can sometimes help you visualise your brand’s tone of voice if you can link it to a fictional character. You can even borrow words and phrases they use in their movies or books to help ground the tone of voice.

It is the Story of your Brand

We all love a good story. Stories are the bedrock upon which society was built, on which all communication was born. Okay, that is quite a hefty statement to say, but we’ve loved stories around campfires for thousands of years. We connect with brands that can give us a good story that we can relate to.

Think of car commercials. While they may tell you about features of the car, you see things like people taking their SUVs to beaches and camping or zipping around the city. You get a narrative of how you can use that vehicle. There is a 4WD drive brand whose ad has no voice-over and no music. It is just two people camping by the ocean, and that is all you hear.

What story do you and your brand have? Why are you in business? Who do you want to help? Where have you come from, your origin story? Where are you going to go?

Brand Voice- why does it matter?

We’ve already said that it is noisy out there on the digital marketplace, so having a tone of voice that sets you apart will help you get noticed. There is a particular seller of tiles on the radio in Australia. As soon as he says “hello”, you know exactly who it is, what they sell, and how you’re going to get bathroom tile bargains. We probably don’t need to mention the brand, and you’re already hearing his voice and know the company.

It Identifies You

Much like the beloved tile salesman from above, your tone of voice identifies you. When you release an article or comment on social media, are people expecting something funny, something bawdy, or well-written, researched, and informed?

Will your readers read your words and feel like you’re a loving parent, an inspirational mentor, or a harsh drill sergeant? None of these tones of voices are wrong of themselves. If you want to build muscle, lose weight and get fit, perhaps the drill sergeant is what you need to read.

It builds trust.

Readers come to your blog, to your YouTube or Social Media channels wanting to see their brands and feel like they’re reconnecting with an old friend. The message you deliver with your content is consistent with who and what you are and what you represent. This builds up trust amongst your followers over time.

This trust can convert into solid sales channels. In marketing, people have to get to know you, like you, and then trust you before they buy from you. If your message is true to your values, and consistent, that trust is built, and your profits will grow.

You easily hit your target audience.

You can sell to anyone, but you can’t market to everyone. You need to find a voice that resonates with your target audience. This can be a little bit of a chicken or egg scenario.

Do you research who your ideal target audience is, research their wants and needs and voice, and service that?

OR
Do you create your brand and a voice to match your values and let the target audience find you?

How to find your Tone of Voice

Following on from above, your tone of voice can begin with your target audience.

Who do you want to buy your products and service?
What are the demographics of your target audience, and is there a language they use?
Are there specific words and phrases linked to your product and service?
Is your tone of voice related to your niche?
Who is it on your team who decides?

Niche audience

If you have a niche audience, this can dictate your tone of voice and make it easier to establish your sound.

Perhaps you own a costume shop. Your clients include theatre companies and some theme parks. But you also have a big Halloween sale every year.

Your audience is in the arts industry, and you can play with theatre directions, mimic The Bard, and much more.

Another target audience could be parents whose children want a costume party. Your tone of voice would be sympathetic, talking about how easier it is to hire a costume than make one.

On the other hand, if you own a car shop that specialises in restoring old hot rods, your audience is into fast cars, hi-octane fuel, shiny chrome, classic cars, and more. Your language would be more ‘masculine’, talking about horsepower, timing, long drives along the coast, and more. It would be authoritative about motor vehicles, but also passionate about loving cars as well.

A good tip when figuring out your tone of voice is to create an avatar of your target audience and imagine having a conversation with them. Alternatively, you build an avatar of your tone of voice and imagine talking in their voice.

How do your audience talk to each other?

Find out where your audience hangs out online, and see how they talk to each other. You can find this audience in many places:

Social media groups.
Comments on popular blogs.
Twitter discussions.
Forum boards.

Does your audience use slang to describe the products and services? Do they engage with any pop culture influences or memes?

Ask your audience.

You can have a survey to ask your intended target audience how they see your brand. You may want to see yourself as a fun kind of brand, but if your target audience needs serious help, you change the tone of voice to be more mature, mentor style.

Audit your content.

Look at the content you have, and review the content you’re going to write.

Does it reflect your values and mission statement?
Does it resonate with your target audience?
Does it convey your message clearly?
Does it sound like how you speak?

Tone of voice Do’s and Don’t’s

Foul language – Understand what you can say and what you can’t say with your tone of voice. While being jocular and using slang may be okay, swearing and other anti-social language would not be okay.

Other brands – Mentioning competitor brands might not be okay for legal reasons, but implying them and comparing yourself to them without blatantly mentioning them could be okay.

Grammar– People like to find grammar mistakes and tell you all about them. Sometimes that can be a part of your tone of voice. If so, be consistent. Otherwise, make sure your grammar and spelling is correct to maintain authority over your work.

Keep it simple – Don’t try to be too over the top creative with your tone of voice. You’re going to have to use it a lot, so keeping it simple is so much easier in the long run. Also, the easier it is to understand, the easier it is for an SEO copywriter to get it right.

Don’t be afraid to change – It is okay to evolve and change over time. What is fun and upbeat in one era may change to more grown-up and mature in the next. Be able to change with the times to capture a broader audience.

Key Take-Aways

You need a tone of voice to stand out from the crowd, to have a personality that people will remember. When you speak with StudioHawk, we hope that the knowledge and fun you read in the blogs and articles is reflected in the Account Managers and SEO Experts you talk to over the phone.

Don’t be afraid to define your tone of voice as differently from others as possible. Don’t be scared to take a few calculated risks to get your message across.

Now you know why you need a Tone of Voice and how it works for you, fill in our Tone of Voice worksheet, to help you define yours and to help our copywriters emulate it.

XML sitemaps optimisation

by Peter Mead - 16 Dec 2021

XML sitemaps are an important part of the Technical SEO skill set you’ll need to know if you want your SEO to go to the next level. In this article, we’ll talk about what is an XML sitemap and how it will help your SEO? How can you build your XML sitemap? As well as how to submit your XML sitemap to Google.

Contents

XML sitemaps for SEO

What is an XML Sitemap?

First of all, let’s talk about what an XML sitemap is. Simply put, an XML sitemap is a file containing a list of URLs on your website that you want to have indexed in Google.

Below is a very basic XML sitemap.

basic xml sitemap example

Why is a sitemap important for SEO?

With an XML Sitemap, you can help Google better crawl, understand and give relevance to the different pages on your website. Kind of like a map telling Google where everything is on your website.

You’re making it easier for Google to do its job by giving it a map to crawl. Google will crawl a sitemap to find all the relevant pages, URLs, and resources on your website. This makes it easier for these pages to be indexed in Google’s search engine.

Do you need an XML Sitemap?

You may not actually need to have a sitemap for your website. Google doesn’t specifically require a sitemap for your website to be crawled and indexed. If your site is smaller with only a few URLs all linked together properly, you may not need to bother with a sitemap. However, it’s a good idea to have one anyway, for SEO reasons.

Keep in mind, this won’t guarantee all your pages get crawled, indexed, or have better rankings. (link:) Google has told us – “Using a sitemap doesn’t guarantee that all the items in your sitemap will be crawled and indexed, as Google processes rely on complex algorithms to schedule your crawling. However, in most cases, your site will benefit from having a sitemap, and you’ll never be penalised for having one.”

If it’s not much work for you, it’s really worth having an XML sitemap, no matter how small your website is. Personally, I’ve seen search results improve when a properly implemented and well-optimised sitemap has been implemented. So, it’s definitely worth having an XML sitemap if you want to see SEO results.

Building an XML sitemap

Check if you already have a sitemap.

Firstly, check to see if your website already has one. Just go to your website, type in /sitemap.xml, and see if the XML sitemap comes up. In our case, we’re having a look at the Semrush.com site. So we’ll type in https://www.semrush.com/sitemap.xml into the browser address bar.

semrush xml sitemap example

Use Yoast to build your sitemap

Using Yoast to build and maintain your XML sitemap is an excellent option for your WordPress website because it updates on the fly as content is changed, added, and removed. This automatically makes sure that your sitemap is up to date, allowing you to give the most relevant information to Google. Later in this article I will explain how to use Yoast to generate your sitemap

Manually create a sitemap

You may need to manually create a sitemap if your website does not have a way to generate one automatically. There are lots of tools available for this. Further in this article, I will explain how to Generate an XML Sitemap with Screaming frog.

Automatically generated sitemaps

If you’re using another CMS like Shopify, or Squarespace, then you’ve already got a sitemap because they have provided you with an automatically generated one. It will usually live in the root directory of your site, /sitemap.xml. You don’t usually have much control over these automatically generated sitemaps.

Submitting your sitemap to Google

Submitting via Google Search Console

Use Google Search Console to submit your XML sitemap. Just go to the sitemaps report. You’ll see there’s a little section where you can type in the URL for your sitemap. Enter your sitemap URL, and then submit it.

Processing your sitemap and checking for errors

That’s it, all done! Oh, wait… You’ll need to give it some time for Google to validate and process the data, of course.

Once processed, this will either show you a little message saying that it’s a success, or if there are any errors, et cetera. You’ll want to keep an eye on the XML sitemap report and check it for errors over time. Fixing errors ASAP to make sure you get the best search results from your sitemap optimisation efforts.

Optimising for better Google results

Clean up sitemaps for a better crawl budget

Cleaning out bloated sitemaps and removing unnecessary URLs can help you get better SEO results by improving your crawl budget. We know crawl budget is a big deal, and if our XML sitemaps are bloated with unnecessary URLs, then it can affect the way that Google crawls your website. It’s definitely worth cleaning up your XML sitemap to improve your crawl budget.

Include your sitemap in your robots.txt file

This helps Google discover it, and it also shows trust and ownership.

Submit using the Google ping service

You can also submit your sitemaps directly using the Google ping service by entering it directly into your browser as shown below.

Directly submit your XML sitemap using the Google ping service. Enter this URL into your browser & replace the value with your own website sitemap URL to your sitemap, and hit enter.

submit xml sitemaps using ping service

Advanced XML Sitemaps Tips

I’ve put together a list of advanced tips to help you quickly make sure you are optimising as best you can and addressing issues as quickly as possible.

XML sitemap priority and change frequency

You can set the priority and the change frequency of an XML sitemap. Something to be noted here is that the priority and change frequency in an XML sitemap may be ignored. Google came out and said they may ignore the priority and change frequency. However, we don’t always just do everything because Google said something.

XML sitemaps guidelines

Please list only your canonical URLs. That means any URLs that are going to a 200 OK, the status code of 200 OK. Only use fully qualified URLs.
Don’t use shorthand and don’t put any other messy URLs in there. Just use fully qualified URLs. Publish your sitemap in your root directory and do not include session IDs in your sitemaps.
Make sure to exclude utility pages, admin URLs, or any other URLs that you don’t want to have found in Google search results.
Including hreflang annotations in sitemaps is also an excellent thing to do if you have a multi-language website.
XML sitemaps must be UTF-8 encoded.
The maximum size for an XML sitemap is 50,000 URLs and 50 megabytes. You can use gzip to save bandwidth. This will help to make Google crawl an XML sitemap faster. You can also use an index file for better sitemap management. Submit your index file in your robots.txt and also in Google Search Console.
Getting a good understanding of Google Search Console sitemap errors will greatly help you optimise your XML sitemaps for better SEO results.
It’s really worth considering the crawl budget, so keep your XML sitemap lean.

Discover your new URLs faster

Perhaps one of the biggest optimisation tips is that XML sitemaps are a great way to help Google discover and index new pages on your site faster. So make sure your XML sitemap is in good order and well optimised.

large sitemaps and index files

What do you do with large sitemaps and index files? You can split large sitemaps of over 50,000 URLs and over 50 megabytes. Split them and use an index file for multiple sitemaps. That way, you can submit many XML sitemaps together. Shown below is a picture of a very simple XML sitemap index file.

xml sitemap index file basic example

Multi-website sitemaps

In this case, we can also use an index file. It lists verified sites’ sitemaps. You can place the index file on one verified website and include your multiple websites sitemaps in that index file. Here, in the diagram, you can see we have five websites.

google multi site sitemaps example

The sitemap’s URLs included in the index file will be stored in the website root that you want to have used for the index file, which contains the other sitemaps. So you’ll need to prove ownership of the other websites. You can do this by placing your index file in your robots.txt file.

You can also go to Google Search Console and verify those other websites and submit them that way. Once you’ve done this, you submit your XML sitemap index file, and this will get Google to crawl and process all of your other sitemaps. You can include URLs from various sites if one file ownership is verified. More here from Google on multi-site xml sitemaps.

Video sitemaps

You can create video sitemaps as a separate file, or you can include them in an existing sitemap. Each entry is for a video on a page. This supplies detailed video information. Google can’t guarantee when or if your videos will be indexed because Google relies on complex indexing algorithms. Here you see an example of a video sitemap.

video sitemap basic example

Image sitemaps

You can create an image sitemap as a separate file or include it in existing sitemaps. Each entry is for images on a page. You can include up to 1000 images per URL. For example, a sitemap with one URL and two images, this is a basic example here.

basic image sitemap example

Google News sitemaps

It’s really helpful and important to have a Google News sitemap if you’re a publisher. You can include articles published in the last two days, and you can remove articles after two days, keeping your XML sitemap fresh and rotating those articles through. News articles will remain indexed for 30 days. There can be up to 1000 URLs per sitemap. You can use the sitemaps index file for larger sitemaps, and you can include 50,000 sitemaps in a single sitemap index file. Here’s another example. Here is an example of a basic Google News sitemap.

basic news sitemap example

WordPress sitemaps with Yoast

A speedy and easy way of doing this, of getting an XML sitemap set up for WordPress, is to simply use the Yoast plugin. Yoast uses a sitemap index file, and this is in the form of sitemap_index.xml. Yoast also updates your sitemap automatically as content changes. Post types that are marked as noindex will not appear in your sitemap.

This is a good way of controlling which URLs appear in your sitemap. The sitemap feature can be toggled on or off because you may prefer to use the built-in XML sitemaps that are now available from WordPress Version 5.5. You can find your XML sitemap by navigating to your Yoast and as included in the picture below.

yoast xml sitemaps

Screaming Frog XML sitemap generator

How do we use the Screaming Frog XML sitemap generator to create sitemaps from a website crawl?

screaming frog xml sitemap-generator

To create sitemaps from a website crawl.

You can include canonical pages with a status code of 200 OK.
You can also include images on a CDN, or a subdomain can be included as well.

To find out more about using the Screaming Frog XML Sitemap Generator go here.

Other Sitemap Generators & APIs

There are many other sitemap generators available and APIs we can take advantage of, far too many to mention them all. But if you want to get started, you can look at Google’s list of (link: sitemap generators as we can see here).

We can also look at Semrush’s list of top 10 XML Sitemap generators.

If you’re interested in the Google API for submitting sitemaps, go here to the documentation.

XML sitemap errors

Many things can go wrong in XML Sitemaps, and a lot of errors can develop, and it’s important to address these errors as soon as possible.

You’ll need to take into particular consideration any errors caused by URL issues and canonicalization issues, as these will prevent crawlers from completing the site crawl effectively.

You can use the Semrush Audit tool to check for and fix any XML sitemap errors. In the graph below, we can see some data provided by Semrush which shows the most common XML sitemap issues. The below data was provided by Semrush and has been taken from 200,000 websites.

semrush audit tool xml sitemap common errors

Here we can see a number of issues:

30.35% of the websites tested had incorrect pages found in the sitemap.xml,
17.41% had a sitemap XML file not found,
84.88% of all websites checked had orphaned pages in the sitemap,
48.7% had a sitemap XML not specified in the robots.txt, and
2.97% had invalid sitemap XML format,
4.45% had HTTP URLs in XML sitemap for HTTPS site, and
0.06 had a sitemap that was too large.

(Table of data)

common xml sitemap issues	Issue type	Sites with errors
Incorrect pages found in sitemap.xml	error	30.35%
Sitemap.xml not found	warning	17.41%
Orphaned sitemap pages	notice	84.88%
Sitemap.xml not specified in robots.txt	warning	48.77%
Invalid sitemap.xml format	error	2.97%
HTTP URLs in sitemap.xml for HTTPS site	warning	4.45%
Too large sitemap.xml	error	0.06%

Putting it into perspective, this data comes from over 200k websites. We’re actually seeing quite a lot of optimisation problems, or opportunities as SEOs. If you can do the work of optimising your XML sitemaps, then you could get ahead of your competition.

Fewer errors with automated sitemaps

Automated site maps tend to have fewer errors, so it’s worth spending the time to get your XML Sitemap automated rather than manually creating it.

Full list of Yoast Sitemap Errors
Full list of Google Search Console Sitemap Errors

It’s worth diagnosing these errors with tools such as Google Search Console, Screaming Frog, and Semrush.

Get started with XML Sitemaps today

XML Sitemaps are an integral and fundamental aspect of Technical SEO. You will want to get a good handle on how to build, submit and troubleshoot them if you want to really kick your SEO into gear.

As I have demonstrated above, there are many ways to approach this, which is typical of many things in SEO. Whatever way you choose to approach this task, it’s worth remembering that you need to do the work if you want to get the results. So dive in and get started, and reach out to us if you have trouble. We are always happy to help.

The Complete Guide to Digital PR

by Lawrence Hitches - 14 Nov 2021

Digital Public Relations (PR) is a strategy to raise the awareness of your brand in the digital space. It is becoming a more powerful way to obtain credible backlinks to help boost your website’s ranking with Google.

Digital PR and Digital Marketing can go hand in hand, but there are subtle differences. Digital PR and SEO is where StudioHawk can help you. This article aims to explain what Digital PR is, why it is an important part of your overall SEO and public relations campaign, and how StudioHawk can help you implement some good Digital PR Strategies.

What is Digital PR?

Digital PR is a tactic used to increase brand recognition online. It is a strategy to get your story out into the world through press releases and publicity, and it is a way to get your content used to produce high-value backlinks to your site.

It is this last element that is now bringing Digital PR into the realm of SEO. Where people would tell you it was a marketing ploy only, using Digital PR tactics for backlink purposes, is a smart and effective way to increase your page authority, your brand outreach, and ultimately your Google ranking.

Examples of Digital PR can include:

Guest blog posts on other websites – this involves writing good content for a website related to yours or posting articles about topics you want traction for. You can get a writing credit and a backlink to your website.
Directory inclusions – adding your company’s website to a directory that lists other similar companies with services such as yours. Include a powerful and attractive blurb.
Press releases – great to send to journalists when you have something newsworthy to say. Feed the journalists some quotes and good information, and they can release a story for you in their publications, with the associated links back to your website.
Unlinked Mentions – while not as strong as linked mentions, any positive brand awareness can have people searching for your website.

Digital PR & SEO

Digital PR is growing in popularity as an SEO tactic due to its effectiveness in picking up backlinks. If you have a great story, and a linkable asset on your website, one press release, sent to enough interested parties, can get you a swathe of backlinks, and a mountain of web traffic.

This will raise your page authority in the eyes of Google, and if done correctly, this digital PR campaign can bring more customers to your website, with the potential for high conversion rates.

Why is Digital PR important to your business?

Digital PR is essential to your business for both brand awareness and SEO reasons.

Brand awareness is getting your brand out there. Think of those big stories from brands that aren’t trying to sell to you.

Nike is a big player in the Digital PR space. They produce inspirational advertising, such as their Dream Further campaign, where a ten-year-old girl steps onto a soccer pitch and imagines her international soccer career. While there was a child-sized Nike soccer shirt being sold, the real success was around 20 high-quality publications online, linking back to the campaign page. There were also close to 2 million views of the campaign on Youtube.

This, of course, converted into sales of Nike branded soccer gear and a raising of awareness.

Traditional PR v.s Digital PR

There are some similarities between the two and some differences. Let’s look at both.

Similarities between Traditional PR and Digital PR

Brand Awareness – no matter the medium, both styles of PR are about raising brand awareness in a positive light.
Positive Messaging – Following on from above, you want to produce a positive message about your company, brand, and any products or services you may be promoting. You always want to deliver a good news story for the press to write about.

Differences between Traditional PR and Digital PR

The main differences between the two types of PR are the communication channels.

Where Digital PR uses social media, blogs, and video platforms to get the message across, traditional PR is more into newspapers and magazines, TV, radio, and events.

Think of product launches with celebrity guests and music. Think radio ads and those advertorial articles you read in magazines. These are the traditional PR methods that have worked successfully for years and still do.

While the world may be leaning more on the digital space, traditional PR still has its place in the mix.

Different kinds of Digital PR Campaigns

There are different kinds of Digital PR campaigns which StudioHawk can run for you. Here are the three main ones, in more detail.

Existing data campaign

This is a campaign where you have content and data, and you want it shared with the world.

Guest posting could include this kind of campaign, rewriting and refreshing an article for a new audience, or writing fresh content for products and services you already have, to generate backlinks.

What is good about an existing data campaign is, the heavy lifting has already been done. You have data and content to share already, and it is now just a process of running a digital PR campaign to make people aware of it.

Survey campaign

A survey campaign is an effort to feel the pulse of your target market. You have a product or service to solve a problem, and you want to know if your target audience would use you if they had such a problem.

The answers you find from such a survey campaign can help make changes to your solution, and it can give you some robust data and subscribers for your list.

Often this PR campaign has free samples, or gifts, to encourage people to try a product or service and to commit to the survey afterward.

Map campaign

Map Campaigns are where you generate data and then use that data in conjunction with a map. Overlaying your data and results with a map is great for Local SEO and for ways to show off how effective your marketing and your brand are at a local level. This effectiveness can be raised to a city, state, or country level, depending on your target audience.

This is good Digital PR because journalists like a good story with some facts to back it up. If they can also talk about localised stories, then they can get a readership at a local level.

How often have you seen stories where a particular suburb has the highest rate of fish and chip consumption? Or the highest housing prices? How about the best plumbers by state? You can make a safe bet that the journalist didn’t research the story themselves but used a Digital PR campaign.

The next great thing about doing some localised Digital PR, and a map campaign, are the backlinks you get from these journalists and other articles which reference your work. Providing good data and a good story will get you good brand exposure and some quality backlinks.

How to Do Digital PR

How can you plot, plan and execute an awesome digital PR Plan? There are four main phases to a good digital PR campaign.

1. Ideation phase

Ideation is a fancy word for coming up with ideas. You need some ideas to work with, which match the topic you have and the channels you can work with.

Some great techniques to come up with ideas include:

Brainstorming– a group of people in a room, throwing random ideas up on a whiteboard. Nothing is off-limits until you sit back and review. It’s good to have one person herding the cats, if you will, so discussions don’t get too out of control.
Worst Idea– What’s the worst that could happen? This helps people realise that some ideas are good because what’s the worse that could happen? A lot of fun too. Sometimes, the worse that can happen could be exactly what you want. Think of those late-night infomercials, where something bad happens, and they provide the solution.
Mind Mapping– is a visual technique for introverted people or those who are averse to adding to an open conversation. Take 5 minutes to write down solutions to the problem. At the end of the time limit, someone at the front adds these solutions to the main problem. You build upon each solution outwards, linking ideas together.
Sketching – for visual people, or artists, sometimes drawing solutions can help. Combine this with a storyboard, and you could map out a great idea into a campaign easily. You don’t have to be a great artist. Stick figures work fine.

Don’t be afraid to mix up the ideas and research other ways to get ideas from people. Make it fun as well. If you work on something you enjoy, the work tends to be so much better.

Choose a topic that is relevant with your business

What does your market want from you? How-to guides and videos? News and current affairs? Ways you can use your product and services which are different? These are called ‘hacks’. Check out the Ikea Hacks community for crazy things you can do with regular stock from Ikea.

The better you know your target audience, your prized customers, the better you will be able to serve their content needs. On the flip side, you can also introduce something entirely new for them, which you know they’ll like and appreciate.

Check to see if similar campaigns have been done recently

Competitor research. You need to see if something similar has been done by competitors in your market or in a new market you are trying to break into.

This research is good for two main reasons:

If a competitor has done an excellent job, you can emulate what they did but do it better. You already have an idea of how successful the campaign can be. Do a better job, give better incentives, get better results.
If a competitor tried something and it didn’t work, you know pretty well that it is not a good idea, and you shouldn’t go down that path.

It’s also a good idea to check if a campaign has been done recently as people have good memories. If your customers see your campaign and think- didn’t so and so do that last month? The impact won’t be as significant.

However, if they remember something similar from the year before, they can feel nostalgic and want to get involved again.

2. Create the Narrative

A good campaign has a narrative, a story structure that a customer can latch onto or they can relate to. Through this narrative, you bring the customers along a journey to salvation, and solution, through your products and services.

Create custom designs for your campaign!

This is the part you get the crayons out. This is when you get to have fun with words and slogans, and catchphrases.

What keywords do you have that can be a part of the catchphrases, the narrative you’re telling?
What colours are you going to use to identify the campaign? Will they be your brand colours or new colours for the campaign?
Do you need a designer to come in and make designs, or can you come up with something yourself?
Are the designs going to be memorable?

It could be argued that the creative process is the most enjoyable part of any marketing or PR campaign. Again, have fun with this stage. If you have fun, then others will have fun and enjoy it too.

3. Build an outreach list

This is the stage you look at what journalists are after, what the news cycles are looking like, where you can land your story.

Not every news outlet or journalist will be a good fit for your digital PR, so don’t try and send it to everyone. The more you know your networks, the better you will be at this stage.

Do your research, have a database of outlets, and what their wants and needs are. It’s also a good idea to develop relationships with journals and journalists. This can make it easier for future digital PR campaigns to get out to the public.

4. Pitching phase

Now it is time to pitch your idea to the journalists, to the public. You’ve got the campaign ready to go. You have assets, such as an image, interview opportunities, graphics, and data, all ready to be shared.

It is a good idea to have all your ducks in a row by the time you send the press release out. If you get a quick response from a journalist, you need to be ready with all the details to run with your story. Do not leave them waiting for you to get back to them. The news cycle can spin pretty quickly.

Digital PR tools

Tools that you can use to boost your Digital PR are third-party options. Here are some of the better ones out there for you to try:

Business Wire – This is a website that has a reach of over 162 countries and over 100 000 media outlets, to which you can release a press release. You need to sign up to see the pricing, but considering the reach this website has and the reputation it has for worldwide journalism and news outlets, it is one of the most important tools in your Digital PR toolkit.
BuzzStream – This is a great little platform for research, pitching, making contacts, getting backlinks, and more. It has a free trial offer, and the basic rate is $24 per month.
BuzzStream is very easy to use and very user-friendly. It has automatic list building, social metrics, contact information from websites, and some great automation rules.
BuzzSumo – Not only can you get access to influencers and marketers and content discovery, but this tool also allows you to monitor backlinks that occur thanks to your digital PR.
Set up a Slack integration and get alerts whenever you, or your competition, acquire backlinks from campaigns. This can allow you to do a better content job than your competitors and possibly win those backlinks for yourself.

As well as these third-party sites, we also recommend using LinkedIn for relationship and network building. Finding journalists and editors there and building rapport with them is a great idea. You can publish articles under your profile and also send out press releases to your new connections. If you already have an active reader base, this could convince those editors to take on your material, as you’d have a guaranteed readership.

Examples of Digital PR Campaigns

If you want to see some excellent examples of Digital PR done right, check out Reboot, a digital PR company based in the UK.

They highlight some of their best campaigns, their favorite being a campaign they made for an energy supplier. They were able to link the amount of harm you were doing to the environment based on what Netflix shows you were watching.

This linked a very topical story – the environment, with arguably the most well-known streaming service in the world. People could relate by identifying the shows they were watching and then getting hard data on how much CO2 was being produced by watching it.

The use of a big brand name got them mentioned in big publications, hence a load of backlinks for their client. The big brand also got them international exposure.

It also had a good news spin by showing how easy it could be for you to reduce your carbon footprint just by watching less TV. Do more by doing less.

Conclusion

As you can see, Digital PR is a valuable tool to have in your SEO tool kit. Not only can it increase the reach of your brand and your values, but it can also bring loads of traffic back to your website. This feeds the bottom line, and it helps raise your page’s ranking.

StudioHawk, a multi-award-winning agency, can get you some big wins through Digital PR and our other SEO superpowers. Check out our DIgital PR page and get in touch with the team to find out how we can get you and your brand noticed on the world stage or in your own backyard.

Building a Culture of SEO in Enterprise Organisations

by Lawrence Hitches - 9 Oct 2021

The covid-19 pandemic has further reinforced the importance of digital marketing to grow a successful business in the current climate. As the CEO for B2C Furniture, I firmly believe that it is absolutely imperative that you build an SEO culture in your organization if you want to future-proof your business.

Ansley Clarke, B2C Furniture

The biggest challenge an experienced SEO specialist encounters when beginning to work with a large enterprise business is making sure everyone is on the same page, whether it be the content creator, business owner, the marketing manager, all the way to the CEO

Why?

SEO is one of the hardest marketing skills to master.

https://www.searchenginejournal.com/microsoft-lists-seo-as-the-most-important-hard-skill-for-marketers/346258/

Being an SEO specialist is hard enough – the skills you need to learn are:

Critical Thinking
Technical / Web Development
Analytics
Creative Problem Solving
Project Management
Communication
Spreadsheets (Lots of Spreadsheets!)

Also be adaptable, motivated and driven……and a sense of humour doesn’t hurt!

“To me, a willingness to try new things from the business makes the best campaigns” –

Chloe Chiang (SEO Specialist)

Being a complicated marketing channel we’re often coming in after a previous agency, assisting an in-house SEO, or even at times a massive company that has never done SEO before. So in execution of these campaigns we make sure we’re building a culture of SEO within the business to develop the strongest campaign possible. We do this through education, collaboration, and building a strong relationship:

Educating Key Players on SEO

To build a culture of SEO in your organisation you need to educate the team on the importance of SEO and best practices. You need to make sure they are willing to put in the time to learn about SEO before coming up with a plan of action, so that stakeholders are onboard from the beginning.

We start with the Fundamentals of SEO – Technical SEO, Search Intent, and Backlinks. In a broad way to all the key account touchpoints, this opens up a dialogue and gets people’s visibility up on what crosses over with SEO.

Then we start niching down into each area of the business, Technical SEO and more advanced ideas we coordinate with web developers. Content creators are run through how to do keyword research, find topics, and produce content their audience is searching for (rather than what they think their audience wants).

Backlink outreach and authority building to marketing managers, and PR specialists is also taught to the teams.

We’re looking to get more eyeballs in the business to find where the roadblocks are or untapped areas of the business that need to be highlighted.

If you’re using super technical jargon, make sure you translate what you mean for those that have no understanding of what accumulative link growth rate, semantic indexing, sitemaps, 301s, and crawl analysis mean.

Always think, TL;DR – too long didn’t read.

Analogies can get the message across and drive through understanding effectively. If it’s an automotive company there’s a billion analogies using cars that work wonders.- get your company into the fastlane, put the pedal to the metal, are just two examples.

Share around your details, email & phone number, be open and available to anyone to reach out with their SEO query, no matter how big or small.

These education sessions can break through red tape and bureaucracy once you get one of those key players hooked into an idea.

Once that little seed is planted, it’ll grow bigger and bigger when the results start coming in. You’ll have more advocates within the business speaking about these SEO wins, which can lead to the next big project.

Guide your client in the development of a culture of SEO and show them how it can help their marketing plans and business’ vision.

Although organic traffic may make up only a small part of a business website’s overall sales, without management and expertise behind it, you’re putting the company at risk.

Through experience in having to educate so much we created Hawk Academy, an SEO learning platform, that way we could get it into the hands of more touchpoints they could complete in their own time.

Collaboration (The IKEA Effect)

We work hard to create an environment where everyone feels like their opinion matters and that they are valued as part of the SEO campaign. That includes our team and their team.

We want people to feel empowered by knowing what SEO is and partaking in the implementation. The IKEA effect is the cognitive bias in which people place a high value on things they partially created.

It makes sense right? Would you feel more satisfied watching someone make a table for you, or would you feel better getting stuck in yourself?

For years we’ve been all for getting our clients rolling up their sleeves when it comes to wins in SEO.

We’re there to be a resource, grow their business, provide solutions, and save their time – but the internal team will understand how to navigate new ideas best and streamline execution.

Collaboration is critical. Rather than assuming one way of doing things is the best – ask the people who deal with the website daily, who understand the complexity of getting through potential red tape.

This can also save a ton of time on campaigns and create a fun working environment with shared wins.

Being “Partners” not “Service Providers”

Clients don’t want to feel like they’re talking to a spreadsheet.

We’re social creatures at heart. While business is business, people want to work with people they like.

Some clients prefer a formal relationship, most however are happy to go out for a few beers or casual dinner to really get to know who they’re working with.

“What we find is that it’s always building relationships and connection with our clients first, SEO comes second” – Joshua Poole (Senior SEO Specialist)

Life isn’t purely work, people have families, interests, reasons why they love what they do. All of this plays into how you can develop more of a partnership. rather than being the people they pay invoices for to do their SEO. It also plays a role in the overall growth of the campaign as they’re people you can reach out to get help to drive more innovation through the business.

Stronger internal relationships allow cross-collaboration and integration with other channels as well, which can increase the chance of an enterprise SEO’s success.

The Complete Guide to Technical SEO

by Peter Mead - 17 Sep 2021

Technical SEO is a large branch of the broader set of SEO with a focus on those items not always seen to the untrained eye. Website SEO-related items that are under the hood, technical items so to speak. This is a huge area of concern, underlying is and supporting large sections of the vast SEO framework.

What is Technical SEO?

what is technical seo

In many ways, technical SEO is the scariest of all the areas of SEO. It quite easily gets the average marketer’s pulse racing, and sweating at the thought of some tedious and complicated issue. However, it’s the SEO foundation, one of the 3 pillars of SEO on which everything else is built. If a website has major technical flaws, then no amount of marketing magic can plug the gap and save your website’s search performance.

Google itself has demonstrated a strong technical culture and has documented the importance of quality technical SEO when ranking your website. Clearly, this is an area every online marketer needs to pay attention to. But what does technical optimisation actually involve?

Why Technical SEO is different

The way you approach Technical SEO is fundamentally different. You need to chunk it down to three main areas of focus to help us think about how to approach technical SEO. The areas of focus being:

Crawling
Indexing
Rendering

Martin Splitt explains crawling, indexing & rendering in this video.

On the surface, these three items may not seem like much, but when you take a closer look, you find that it really is a case of how far down the rabbit hole do you want to go? This approach is focused on how search engine bots crawl, index and render your site, and this usually involves those more technically focused issues within your website.

Crawling, Indexing & Rendering

It’s vital that search engine bots can find your page and crawl it effectively, including your website’s status codes, sitemaps, architecture, robots.txt.

You want to make it as easy as possible for search engine bots to index your web page. But any unwitting methods used could cause your web pages to be non-indexable.

How search engine bots render your webpage is essential if you want to be indexed effectively. Consider how they may be rendered with JavaScript, the DOM elements, and page speed.

Recommended reading: JavaScript SEO

As your page renders, it’s important for the content to be understood if you want the best outcome indexing. You need to consider how well you have implemented schema markup, and how accessible your web page is. Sending the right signals with canonical tags and effective pagination is essential

Technical SEO baseline

Technical SEO spends a lot of time obsessing about the need for Google to crawl, index and render your web pages effectively. If this technical SEO baseline is not lived up to, then you will find your efforts pushing up against some boundaries, and you won’t be able to burst through that ceiling for your rankings and desired traffic.

Recommended reading: Getting started with technical SEO

Understanding HTTP Status Codes

Search engine bots use HTTP status codes when crawling to help determine the health of your website. These Hypertext Transfer Protocol (HTTP) status codes are given by a web server in response to a request made by the client or a crawler when you browse the website, or when search engine bots such as Googlebot, crawl the website.

Common HTTP Status Codes

You will want to enhance your technical vocabulary by understanding HTTP status and error codes. Below is a list and an explanation of each of the most common error codes

Common HTTP Status Codes are:

200 (OK) – This is the response you want for our pages to be indexed by Google.

301 (Moved Permanently) – Tells Google our page has moved to a new location.

302 (Found) – Ambiguous, but thought of as a temporary redirect, not permanent.

307 (Temporary) – Temporary redirect, this replaced the 302, is now more precise.

403 (Forbidden) – There is no proper permission to access the requested resource.

404 (Not Found) – The page can’t be found resulting in a broken link.

410 (Gone) – Use this to indicate to bots the page is permanently removed.

500 (Internal Server Error) – An unwanted, unexpected error has occurred.

Further learnings on HTTP status codes

The full list of HTTP status codes is quite long and very involved. Here you can see Google has a Status code resource for everyone to read and reference. It’ll be a great idea to get to know these a little and understand how they work.

HTTPS and SSL for securing your website

Internet security in this stone age is vitally important. Google has made it clear that they take Internet Security very seriously, takes action on unsecured websites.

What is HTTPS

Hypertext Transfer Protocol Secure (HTTPS) is an Internet protocol designed to protect the identity and confidentiality between the computer and the website. Users in this day and age expect privacy and security when using someone’s website.

Why HTTPS is important

There are still many websites that do not have HTTPS and are still using HTTP. We know that Google prefers HTTPS, and our SEO efforts need to ensure we have implemented an SSL certificate correctly.

You need an SSL Certificate

You need to have an SSL certificate if you want to enable HTTPS for your website. Make sure you get your certificate from a trusted provider. Your web hosting company will most likely be able to help you enable your SSL certificate. You can also speak with a web developer to help you implement https on your website.

If you are updating your website from HTTP to HTTPS then this is considered to be a migration. Google will treat this as a site move with URL changes. The migration can affect some of the traffic temporarily. It’s important to treat this as a migration and make sure the necessary redirects are put in place to avoid losing too much traffic.

XML Sitemaps & informing Google about your URLs

An XML sitemap is a file on your website that gives information about the pages, videos, and other files and resources. XML sitemaps help search engines like Google read and understand the files and understand the relationship between all these URLs.

It’s a way to tell Google which pages are important on your website and it’s a way it also helps Google with further information about these URLs.

You can also provide sitemaps with information about more specific types of content such as videos, images and news.

Not every site needs an XML Sitemap

If you’re using a common content management system (CMS) such as WordPress or a platform such a Shopify, you’ve probably already got an XML sitemap provided for you. You may not need to do much in the way of working on your XML sitemap depending on your website nature.

Not every website needs to be overly concerned about having an XML sitemap. For example, if all your pages are properly interlinked and if Google can crawl and discover all those URLs, for smaller sites, you may not ,strictly speaking, need to implement an XML sitemap.

However, since you want to get better results through SEO, it’s highly advised to implement a well-structured XML sitemap on your site.

Get started with XML Sitemaps

The most basic XML sitemap format Specifies that a site map must be limited to 50 megabytes in size and include a maximum of 50000 URLs. If you have more than 50,000 files on your site, you can break your sitemaps into multiples and use a sitemap index file.

A sitemap index file is a list of sitemaps and you can submit the index file to Google, which then tells Google where to find all the other site maps on your website.

Below is an example of what the most basic XML sitemap looks like.

basic xml sitemap code

(image credit: Google Search Central)

Recommended reading: Learn more about XML Sitemaps

Robots.txt file – which pages can be crawled?

Robots.txt is a file that tells search engine crawlers which URLs on your website it should access and crawl. It helps Google understand which pages not to crawl so that you don’t get too many crawl requests for URLs on your website.

You can use a robots.txt file to manage crawl requests to your server to avoid too much web server traffic. It’s common practice to block resource files and unimportant images and scripts from being crawled.

Particular attention should be paid to which resource files you are blocking because some of these files may be important for Google to crawl and understand the webpage, particularly during the rendering process. So Google needs to access all relevant files and resources on your website so that they can get an understanding of your site and index it effectively.

It’s a common mistake that people think using robots.txt is a way to keep web pages out of the Google index. Disallowing URLs in your robot.txt file is not a reliable way to do this.

If you want to keep certain pages out of the Google index, you will need to block the indexing of those pages by using a noindex directive, or you may need to password-protect that page.

Below are some basic robots.txt examples.

This first example tells all the crawlers that they can freely crawl the website because of the wildcard * and the Allow.

robots txt allow all

This second example tells all crawlers not to access any pages on your website due to the wildcard * and the Disallow.

robots txt disallow all

You can tell crawlers not to crawl certain directories.

robots txt disallow directories.

Specific files to be disallowed can also be specified.

robots txt disallow file

It’s also important to include your XML sitemap in your robots.txt file. This helps Google find and verify it. Include it in one line, like shown below!

You can use the Studio Hawk Robots.txt tool to test and validate what’s in your robots.txt file.

Meta tags for search appearance and behaviour

Meta tags are an excellent opportunity to tell search engines more information about your website pages. Meta tags are placed within the section of the HTML code of your page. You can use them to explain to Google what our page is all about and what to do with the page that it’s crawling.

The most important meta tags are:

Page title
Meta description
Meta robots

Page title tag

The Page title is not a meta tag but is considered one for convenience sake. It’s actually a title tag placed within the head of the html code.

Here is an example of the Studio Hawk home page title.

The title is very important for SEO because it tells users and Google the name of your page. If implemented correctly, your title will appear in Google search results as the big blue line of text, see below.

title tag example in serps

(Note: Google is now also swapping out title tag with the H1 tag in search results.)

It’s often the deciding factor for whether a user clicks your result or someone else’s search result. Your title tag should give users a look into what’s on the content of your page and how it’s relevant to their search query.

You want your titles to be unique and to be brief and descriptive. You also want your title to include your target keyword if it makes sense in a readable way, and it should match the users’ search intent. It’s best practice to keep your titles under 60 characters long.

Meta description tag

The Meta description tag gives a summary of what’s in that page’s content. Use it to place a short and accurate description of what your page is all about.

meta description code example

It’s often used by search engines for the snippets section of the search result as can be seen below.

meta description example in serps

Meta description tags are an excellent way to give information to the users with a relevant summary of what that page is all about. It’s a bit like having a short pitch that makes the user want to click on your result and go to your page.

It’s good to optimise these meta descriptions to encourage more people to click through from the search results giving you more traffic to your web pages.

It’s important to know that meta descriptions are not a direct ranking factor. Even though Google does sometimes use the meta description for the snippets section of the search result, Google does not use the meta description as a part of its process for rankings.

You’ll want to write unique meta descriptions for every page on your website. These meta descriptions should be an accurate summary of the content on your page. It’s an easy trap to fall into writing generic descriptions across your pages due to a lack of time to write high-quality unique meta descriptions.

Make sure your meta descriptions are not clickbait, but something that’s click-worthy, and matches the users’ search intent. You’ll want to include your keyword in your meta description where it’s reasonable, while making sure it’s easily to read. Best practice is to keep them under 160 characters long.

Meta robots tag

You can use the meta robots tag to tell search engines if they should crawl and index your web pages.

The above example tells search engine crawlers to crawl and index the content and links on your page. This is a vitally important tool for SEO to control which pages are indexed and which pages are noindexed:

index – you want bots to index the page.
noindex – you don’t want bots to index the page.
follow – crawl the links on your page, and they are endorsed.
nofollow – don’t crawl links on your page, they are not endorsed.

For example, you could tell search engine crawlers not to follow or index the page by placing this meta robots tag in your HTML code:

The above example tells Google not to crawl the links and not to index the content on the page.

But it’s important to know that Google treats nofollow as a hint rather than a directive. So even if you’re using nofollow the Googlebot Crawler may still make up its own mind.

If we get our meta robots tags wrong, it can seriously impact how we appear in search results. You can remove pages entirely from Google search. So it’s best practice to edit our robots tags when we want to restrict how Google crawls our page.

301 Redirects

When it comes to redirects, it’s important to have a good strategic approach for using them. You want to present a coherent picture to search engines. For example, if a page has been removed and it’s gone forever, it’s better to use a 301 redirect to transfer the ranking signals to the new destination page.

You’ll only want to use a 307 redirect when the page has been moved temporarily and after some time, you’re going to put that page back.

Redirect chains

A loose redirect strategy can lead to multiple redirects from one page to another, known as redirect chains. This is where crawlers will access the page to be redirected to another page, and then that page also redirects to another. This causes big issues for search engine crawlers and affects your crawl budget and your canonicalization.

Here’s a diagram of a redirect chain:

Page A redirects to Page B, which also redirects to Page C, and then to Page D, causing multiple hops to take place.

Removing redirect chains can have a very positive effect on your crawl budget. Below we can see the redirects chain removed and simply redirected with a single hop.

unchaining redirect chains

Remove old redirects

Clean up any old redirects that have been put in place but are no longer used. In the above example, Page B may no longer be in use, and it may not pass any ranking signals either. There’s a good chance it was put in place and forgotten when the page was moved again.

In this case, it may be better to remove the old redirect from Page B to Page D altogether, and clean up your internal links, so they all point to Page D.

Fix Broken Links to reduce 404 errors

If your internal links are causing 404 errors, the page that’s linked to can’t be found. That is called a broken link. Broken internal links can indicate quality issues and a lack of website maintenance. Broken links to external websites are perhaps not quite as detrimental.

In any event, it’s very annoying for users, so causes a user experience issue. And if it detracts from the user experience, then good bet Google will also take it seriously. Unaddressed broken links can eventually lead to a loss of rankings.

Finding broken links

Using a link checking tool such as Xenu or Screaming Frog will help find broken links and flag them as 404 errors for you. It’s worth scheduling a regular crawl with one of these tools. You can also see 404 errors in your Google Search Console.

404 errors can become out of control quickly, particularly on large sites.

Fixing broken links

Addressing incoming external broken links is also vital to your SEO efforts. As SEOs, we put a lot of effort into getting a good backlink, and if the page you’re linking to cannot be found, then the benefit of that back link is lost.

It’s worth regularly checking your incoming backlinks and redirecting them to a similar page if the target page is no longer available.

Use a tool like Semrush Backlink Audit to find good quality backlinks pointing to missing pages on your site. Then look for the next most relevant page, and redirect the backlink to that.

Improve the quality and trustworthiness of your site by finding and fixing broken links and reducing the number of 404 errors, and you will see the benefit in your search results.

Core Web Vitals & Speed Optimisation

Slow page speed is one of the oldest and most consistent issues found by SEOs when optimising websites. Sluggish performance can annoy your users, but also it can seriously affect the conversion rate when the user goes to another competitor’s website which may run faster. This causes a bad user experience.

Speed is of particular importance on mobile devices, due to the mindset the user is in while looking up information on their hurried lunch break or the train commuting. Also, mobile internet may not be so reliable in certain areas, which may cause a speed delay to begin with.

So the mobile version of your site needs to be lean and run fast.

Recommended reading: How to speed up your website for mobile

Google measures the speed of your web pages using the Core Web Vitals metrics. These metrics are designed to measure the kinds of elements that will affect user experience. Each of the Core Web Vitals (CWV) addresses a particular measurable area of how users find your web page experience.

Google breaks CWV down into three main areas:

Loading
Interactivity
Visual stability

core web vitals lcp fid cls

Largest Contentful Paint (LCP) – is used for measuring page loading. The LCP should be within 2.5 seconds for a good user experience.

First Input Delay (FID) – is used for measuring the interactivity as the page is loading. We need a FID of less than 100 milliseconds for a good user experience.

Cumulative Layout Shift (CLS) – is used for measuring the visual stability of your page. We should have a CLS of less than 0.1 for a good user experience.

You can quickly check your CWV for any given page by using the PageSpeed Insights tool, It will give a breakdown of the CWV metrics.

Below is Google.com as measured with the PageSpeed Insights tool:

page speed insights core web vitals score

Google is taking speed very seriously and have included a dedicated report for CWV in Google Search Console which you can take advantage of to improve the speed of your pages.

The report measures URLs and labels them as Poor, Needs improvement, or Good, for either Mobile or Desktop. It also gives example URLs so you can look them up and action them.

Large and bloated images

Bloated images are one of the main causes of slow page load times. It’s recommended to take the time to compress your images before uploading them to your site. Make them only as big as they need to be for the page.

Lazy loading images

Lazy loading is another way to reduce the initial load time of a page. Lazy loading means any images below the fold will only load as the user scrolls down the page. This means that all the images on the page didn’t need to load all at once. They can be loaded only as needed as the user scrolls down the page.

Defer loading of JavaScript

This is a useful way to improve FCP times by only loading hefty scripts when the page has finished loading. Some JavaScript can be quite large and if they are loading at the start of the page load, it will slow down the whole page load.

This is because browsers execute the page code from top to bottom and load the resources as they find them. Deferring loading of JavaScript can speed up FCP load times.

Remove animated elements

Although you may think that your users may appreciate it when that button text slides all the way in as the page loads, think again. The time that it takes for these elements to load affects your CLS score.

Google has said that user experience is hampered by these kinds of elements rather than improved. Consider removing these kinds of animations to improve your CLS.

Recommended reading: Why page speed is important for SEO

Mobile optimization & Cross-Device Compatibility

Back in the early days of the web, trying to make sure your site rendered well and worked properly in the two or three main web browsers was a real challenge. Thanks to the maturation of HTML standards, these cross-browser problems are becoming a thing in the past.

Why cross-device compatibility is important

Today web designers have the difficult task of ensuring the site works well on desktop computers, tablets, smartphones, smart TVs, and any other class of device you can think of. The more cross-device compatible your site is, the easier it will be for a search engine to make sense of it and rank it appropriately.

Test your site on as many real devices as you can, and use online emulations to cover the more esoteric possibilities. While it’s almost impossible to achieve perfect functioning across the whole range of devices, at least make sure that your site degrades gracefully on an incompatible device, rather than collapsing into an unviewable and unrankable mess.

Using Google’s Mobile-Friendly Test

Testing your site regularly with the Google Mobile-Friendly Test is a must. As an absolute baseline, you want your site to be what’s called “Mobile-Friendly”. Google believes the future is mobile-based, with the web being accessed more and more on mobile devices, so they want to cater more for that.

mobile friendly test result

Google made the importance of mobile abundantly clear when they introduced “Mobile-First Indexing”, meaning that the mobile version of your site is crawled and indexed 1st. A strong technical SEO focus on optimising the mobile version of your site will go a long way to getting you better search results.

Canonicalization & Duplicate Content

Avoiding and disambiguating duplicate content is an important part of technical SEO. It can affect your rankings when Google is confused by which piece of content is the original and preferred.

Although copying large chunks of content onto multiple pages is an obvious no-no, it’s also a problem when poorly thought-out or badly implemented URL schemes allow several URLs for the same content.

Google says- “Duplicate content generally refers to substantive blocks of content within or across domains that either completely match other content or are appreciably similar.”

Why duplicate content is a problem

Search engines cannot tell which version of the page is the preferred version, and so the bots cannot choose. Unless otherwise specified, many duplicate versions of the page may be indexed in the search engines. This makes it very difficult for search engines to know which version of the page to rank for a query.

You should have a preferred version of the page and that’s the one we should be linking to both internally and externally. Linking is just one of the signals Google uses to help determine rankings. But if you are also linking to other versions of the same page, you are watering down your signals and confusing the search engines.

Finding your canonical page

You can use URL Inspection Tool the within Google Search Console to find which page has been chosen as the canonical. At this point, you may be surprised to learn that Google could have chosen a different canonical than the one you desired.

There are varying reasons for this based on signals, such as incoming links to the page, the page’s performance and engagement metrics, and a range of other indexability and ranking considerations.

Finding duplicate pages

You need to find those duplicate pages that have begun multiplying across your site. Particularly if your site has been around for a while, then these pages could be creeping up over time. You need to find these so you can get a clear picture of just how many duplicates you have so you can take action to address them.

You can find duplicates by doing a Screaming frog crawl, then go to the “content” tab and select “Exact Duplicates” from the list selector, as shown below.

exact-duplicates in screaming frog

Many of your duplicate pages may not be exact duplicates per se but may be near duplicates. This means a high percentage of the content is duplicated, and therefore the page is still considered duplicate content.

The threshold for near duplicates can be adjusted under Configuration -> Content -> Duplicates. Here you can enable near duplicates and set the threshold. See below.

near duplicates in screaming frog

You can also look at a list of all your current canonical links by going to the Canonicals tab and then looking at the Canonical Link elements. Also, pay attention to how many Occurrences of the canonical there are. There should only be one per page, as shown below.

canonical tags in screaming frog

Using the canonical tag

The proper use of the rel=canonical attribute is a great way for handling duplicate content, by telling Google which page we want to be indexed. It’s a good idea to have a self-referencing canonical on every page that you want indexed. For any duplicate pages , you place the canonical, so it references the original and preferred page.

You can also place a cross-domain canonical reference for your content which is duplicated on other websites.

When used correctly, the canonical can help signal to Google that a given page is only a copy and that the rankings should be attributed to the original page. To see what a canonical tag looks like, browse to a page on your site using Chrome, then hit Ctrl+Shift+I to inspect the page with Chrome Dev Tools.

Then search for the word “canonical” this will highlight your rel=canonical in the source code of your page.

canonical ref code view

If the URL in the rel=canonical points to the preferred page you want indexed, you’re good to go. However, if it’s not pointing to the correct page, you will need to address this issue.

The power of the rel=canonical should not be underestimated, but it’s important to remember that setting the canonical is a signal, not a directive. Meaning, Google may still determine that another page on your site should be the preferred canonical.

In this case, there are very likely other signals overriding your user-declared canonical. It’s really important to update all your internal and external links to the intended canonical page.

301 Redirect unwanted duplicate pages

There comes a time when you realise that maybe many of the duplicated pages are just not serving you in any way. Once you realise this, you may decide to remove them from your website. But now, you may have created another problem – broken links. You removed the page, but internal or external links are still pointing to it, causing a 404 Page Not Found.

In this case, it’s a good idea to place a 301 redirect and point it to the next most relevant page on your site.

Internal Linking for structure and discoverability

Search engine crawlers or spiders navigate the web and discover your content via links. Having a good internal linking strategy gives structure and crawlability and helps your content be found. This also provides structure to help pass link metrics to the most important pages on your site and help improve your rankings.

What are internal links

Simply put, it’s a link from one page on your site to another page on your site. These links help people and search engine crawlers to find and navigate to pages on your site. It’s absolutely essential to have internal links to the page that you want to be found for.

A page without any internal links will be harder for Google to index and rank.

There are two main types of internal links, depending on how your site links are implemented.

Types of internal links

Navigation links – placed in your header, footer, menus and navigation bars, to help people navigate.
Contextual links – placed contextually within the page’s main content , these pass link value and relevance.

Navigational links are usually pretty static across the website, and they mostly do point to important pages on your site. However, navigational links can be considered as boilerplate and therefore not offering as much value as we may have thought.

We can be much more strategic with our internal linking using contextual links, which are highly relative if thoughtfully placed from one page to another relevant page on our site.

Below is an example of a contextual link within the content of a page, linking to another related page on the website:

contextual internal link example

How internal links pass value

Google looks at the number of internal links and their quality to a given page as a signal of importance and value for that page. Since you have complete control over your internal links, you can ensure your most important pages have a good number of quality internal links coming to them as a site owner.

This helps Google understand the relationship between your pages and how your page fits into your overall structure. Google gets an understanding of your site and the important pages by crawling all your contextual links.

Therefore we can help search engines better understand which pages are relevant and more important by placing contextual links.

How links pass value can be ultimately understood by looking at the PageRank algorithm by Google. However, in this article, we will look at how value or equity is passed as it is very similar to PageRank.

It’s an interesting journey if you’re up for a deep dive into how PageRank works on Wikipedia The below diagram demonstrates how equity may be passed.

how internal link equity pr is passed

The link equity passed by contextual links is divided by the total number of links on a page. This means you can overdo it by placing too many contextual links on a page and watering down the value of those links that the equity is divided among.

how pr is distributed internal links

Usually, the most amount of link equity can be passed by linking to other pages on your site from your home page. This is because the home page usually has the most incoming backlinks from other sites, and we can then distribute that link equity thoughtfully to other pages on your website.

An example of this is powering up a newly published blog post by placing a link to it from the home page of your site. This will take the link equity of all your incoming backlinks and share it out between the links on the home page, including the link you placed to your new blog post.

Internal linking strategy

You’re going to want an internal linking strategy to make sure your most important pages get the most benefit from your internal links. Keeping in mind the reason for our internal links, we should consider these three determining factors for creating internal links:

Factors for determining internal links:

Relevance between pages
Relationship between pages
Value passed between pages

It’s helpful to think of your website as a pyramid structure, with your homepage at the top. The layers underneath will be various categories and sections within your site. Layers further down will be the individual services, product pages, and blog posts etc.

You will not want to create too many layers. Just keep it at 3-4 layers deep maximum for best results.

internal linking structure

You will want to focus on how this pyramid structure is weighted with links to your most important content. The more important the content, the higher the ratio of internal contextual links to those pages.

For example, if you’re writing lots of articles about a flagship product, then each of these should contain a contextual link to the definitive page for that product. The idea is to concentrate those links from the most relative pages to the most important page on that topic.

Also, place a link from this page back to the articles to help reinforce the structure, and spread link equity by relevance.

You’re going to want to put a fair effort into this internal linking strategy. Keep at it, making sure all your content is linked strategically, particularly new content which may get published and forgotten about.

Develop your internal linking strategy because it’s one of the most powerful ways to help Google help you get better search results.

Your Technical SEO journey

In essence, technical SEO aims to remove barriers from the search engines’ path, making it as easy as possible for them to understand your site and rank it well. I have always taken a strong focus on Technical SEO when working on client campaigns, which has always yielded terrific results.

Dive into crawling, indexing and rendering to make sure Google sees and ranks your site. Get under the hood with HTTP status codes, and configure your HTTPS and SSL. XML Sitemaps and Robots.txt are absolutely essential, along with Meta tags.

Work on your 301 Redirects, address the broken links, and sort out your Canonicalization. Optimise the Mobile version of your site, make sure it runs fast and passes the Core Web Vitals. Never underestimate the power of a good solid Internal linking structure.

Happily, making your site more technically appealing to Google also makes it more useful for humans – the same efforts will pay you back twice by boosting rankings and increasing conversions.

The Complete Guide to SEO Content

by Harry Sanders - 19 Jul 2021

A well-thought-out and considered content strategy is crucial if you want some big wins with your SEO strategy. Content is also something you can do without a load of technical knowledge and special apps or software. All you need is an idea of what makes good SEO content. Writing content is good, writing SEO copy is better.

SEO content is content that has the intended purpose of helping your web pages rank in search engines for user queries. It’s a key part of how SEO works. Content is one of the 3 pillars of SEO and includes everything to do with the writing and structuring of content on your website. There are three major elements you need to consider to produce content that will make your website rank well: keyword research and strategy, having an optimised site structure and page structure and copywriting.

SEO content is arguably the only content you should have on your website. Every piece of content online should have a purpose, from blog articles answering sought after questions, images with alt tags to show off your products and services; product pages with copy at the top of the page to both explain to the user what the products are all about, but also to help the search engines index and recognise what is on that page.

This article is going to present to you the full range of SEO content, what it is, the different kinds of content included in this category and why SEO content is important to the success of your website. We’ll also go through how to get started with putting SEO content on your webpages.

Why is content important for SEO?

SEO content is important for several reasons:

Context- The more content you have for the crawl bots, they can get a better idea of what your website is about.
Visibility- This means more chances for your website to be delivered in a SERP.
Immediacy- You can begin to target new keywords quickly and easily with fresh new content.
Activity- new content updated regularly shows Google that your website is active, so they could prefer returning search results from your website as opposed to one which hasn’t been updated in months.
Building Brand Awareness– If your brand is higher up the rankings on a SERP, the impression by users is that you can be trusted, and Google thinks your website can give them the best answer.
Social Sharing– With good content drawing organic search users, if you have social share buttons, users can easily share with their social networks. This can bring more people back to your website, increasing the good metrics.
Increase Domain Authority- If all of the above helps bring more people to your domain, the increased traffic tells Google that the content you provide must be good. This can raise your domain authority and hence your ranking with Google.

SEO content feels like an all-encompassing phrase. There are many different kinds of SEO content you can produce, which serve a different range of SEO purposes.

Blog Posts

Blog posts are the SEO workhorse of content and are what most people think of when content is mentioned.

A blog can be packed with SEO keywords, internal links, embedded videos and more aspects of good SEO. They can tell the story of your products and services, how they help other people and how they can help you.

Blog posts can also be used in multiple ways and are great value for money.

Product Pages

Products drive cash flow. They service transactional search intent. Strong SEO content on product pages increase the chances of your products being found, and increases the chances of someone adding your products to their shopping carts.

SEO content for product pages can include above the fold copy, with keywords associated with the products. It can include Product descriptions with FEATURES and BENEFITS listed for clients to be enticed.

Properly optimised, product pages will be indexed by Google and served up for transactional search intent. Clear SEO content makes this process easier.

Service Pages

Similar to product pages, instead you are creating content for services offered.

Have the FEATURES and BENEFITS written for the user to understand. Have the page optimised for the crawl bots to index and understand as well.

Category Pages

Product and Service Pages sit underneath a Category Page. For example, the category page could be all about sports shoes, with subsequent product pages for basketball shoes, football boots, tennis shoes and so on.

Think about the search intent. Some people may search for tennis shoes, while some will just search for sporting shoes. Linking to your products from the category page is a powerful SEO strategy. The craw bots can follow the logic from Category to Product, and as such help rank you for your products.

Articles

Similar to blogs but with a more formal tone. These are still SEO compliant, with keywords and a structure which the search bots can understand, and often the word length is above 2000 words.

Another function of the article, they can be submitted to news agencies and magazines. Include a backlink to your site and you begin to raise the Page Authority of your website, and hence the ranking results in SERP.

The more niche you can get with your articles the more SEO power the content will have.

Lists

Often called Listicles, these are blogs framed as a numbered list. The top ten cookies, 7 ways you can paint a house. It is literally a list to help users out.

There is a higher chance your list can appear as a SERP Snippet, the zero-click text that appears at the top of a SERP.

Guides

Guides are longer form content with the intent to help users understand a topic or a concept. A lot of time and research is invested into a guide so that it can be the ultimate resource and reference point.

They follow the same ‘rules’ as a blog, with keywords, title tags. You can also hunt backlinks to point to your ultimate guide to increase the ranking on search results.

It is also advised to have a downloadable PDF version to enhance the user experience.

Infographics

This is SEO content which caters to people who digest more information through visuals rather than just text.

Infographics are great pieces of SEO content to download, to share onto other sites, and to catch the eye with how the information is presented.

Glossaries

A good SEO tool to collect keywords without being penalised for keyword stuffing. Also good for internal links when a definition is required.

Videos

SEO content which is colour and movement, serves people who don’t want to read text. It can be shared with YouTube as well, the second largest search engine after Google.

SEO Keyword research

Keyword research is important so you don’t waste the writing you’re doing. The power of your content is significantly weakened if you’ve done no keyword research and not included keywords in your work IF you’re going to spend the time creating content you want it to work for you.

What is keyword research?

Every content SEO strategy should start with keyword research. Discovering the specific words and phrases your target audience is searching for helps to generate organic traffic to your site.

It is finding the words which are key in helping you rank high on search results pages.

Research can help you:

Find words that you want to rank for
Find gaps in the market where you can take advantage of by using certain keywords.
Help you discover if competitors are using certain words, so you can compete with them.
Attack new words with your new content so you can rank for those searches.

Keyword research has four main steps:

Write down the mission/purpose of your business.
List all the keywords you want to be found for.
Look at the search intent of people searching for products and services similar to yours.
Create landing pages which serve search intent and include your keywords.

If you do your keyword research right you will have a clear overview of what search terms people use and the search terms for which you want your pages listed for.

This overview will serve as a guide for future SEO content on your website.

Why is keyword research important for SEO content?

Conducting keyword research identifies what search terms you customers are searching for, so you can align the SEO content to match.

The words and phrases you use to describe your products and services may be different to what your customers use. Remember you are trying to rank for any search term your customers use, not what you use. You are serving their search intent, not yours.

If you owned a gardening business, some industry keywords could be fruit trees, vegetables, fertiliser, and so forth. However, people may be searching for – when should I plant tomatoes? They already know you sell fertiliser, so why search for it?

Keyword research can save you time and stop you trying to rank for keywords which your customers don’t use.

What is a keyword?

A dictionary definition will tell you that a keyword is a word or concept of great significance. This is a good definition and it fits into what keywords are when it comes to SEO.

They are the target words which connect a search query with a search result. If you were looking for a new leather jacket, the ‘leather jacket’ would be the keyword. Why aren’t they two individual keywords? Because of context and search intent. You don’t want a target keyword of “leather”. That could branch off into jackets, couches, belts, and all sorts of things. Hence, the keyword is leather jacket.

You would use these words in the content on your website, to attract users via organic search. You can also target these two words in a ‘long tail keyword’, where someone uses a phrase – “new black leather jacket” or “leather jacket worn by TV star in 50’s sitcom”.

Variations of these keywords can be targets on your website, and as long as your content and metadata include the keywords, you will have a chance of being returned as a search result.

What is long-tail keyword research?

This is slightly different from just keyword research in that you’re researching phrases people are looking for, rather than single words.

An increasing number of search queries are using long-tail keywords and keyword phrases as search algorithms become more advanced and gain a greater understanding of user intent.

Understanding these phrases, and including them in your SEO strategy can greatly increase your chances of ranking based on search intent.

Long tail keywords which fit your niche can highlight your products and services. If you compete for low-scoring niche keywords, you can potentially get some good results in a good time. You will also attract customers who are already warmed up to your ideas and are passionate about who you are and what you have to offer.

How many keywords should you focus on?

How long is your piece of string?

You need to focus on enough keywords that you can get results, but not so many that you stress over trying to win ALL the words.

A few hundred is a good number. This gives you opportunities to work on some keywords now and have some ready to move to when you start another campaign. It’s always a good idea to be working on SEO. In the SEO game, being idle is bad.

What is search intent?

In simple terms, search intent is what a person is searching for, the intent of their online query. Are they looking to buy something? Is someone googling for directions?

Google will serve up search results based on INTENT. Knowing what a user’s search intent is and creating content for it is good for business.

What are the 4 types of search intent?

Informational. People are searching for the answer to questions. “How to” and “what is” indicate informational search intent. If you can answer questions, based on your product or service, then you can possibly rank with informational search intent.
Google knows more than just what the search intent is for thanks to all the data they’ve vacuumed up about you. For instance, if you asked ‘how do I grow tomatoes’, Google knows you’re into gardening and will widen the SERP to include broader gardening tips if there is nothing specific enough about tomatoes.
Google will also include video content if it is available, to shake up the different kinds of content available to answer the intent.
Navigational. When someone wants to visit a website but can’t remember the actual URL, they will search the name of that company or website and click through.
Ranking for this intent works if you’re the business people are searching for, but this search intent ranking doesn’t have a long shelf life.
Transactional. A high volume of online traffic is transactional where people are searching for something to buy, to enact a transaction. This kind of search intent exploded during 2020 with online shopping booming thanks to lockdown.
These users are looking to buy, if you can service this search intent and make buying from you easy, your cash flow can look very good.
Commercial. These are people planning to buy. They are comparing products and prices, matching businesses up against each other.
Writing reviews or articles of comparisons, opinions about which product or service is best, and why, would service this search intent.

Website Structure

Thinking that merely having a website with content online is enough is a limited view. You can optimise the structure of your website to help serve the web crawlers which come to index your website for the best search results.

Organising your content into ‘silos’ helps crawl bots understand what content links to what, how much content you have on a certain topic, all of which adds to the chances of your web pages being returned in search results.

A good example of this are retail stores. Imagine an online store which sells Jewelry. Ear rings, necklaces and rings.A bad structure would be to have all your jewelry listed on one product page.

A Better structure would be to have three product category pages, one each for earrings, necklaces and rings. From these pages you then link to different styles of each. To keep it all organised, you wouldn’t link to necklaces from the ear ring silo.

By doing this, Google knows firstly that you’re a jewellery store. Secondly you sell three kinds of jewellery, and thirdly you have all these products under each category.

What is the ideal structure of a website?

website-structure

The ideal structure for your website is a hierarchical pyramid type structure, where you have content silos, and no more than 3 clicks to get to the information you’re looking for.

The top of the pyramid is your home page.

The next level of your pyramid is your CATEGORY PAGES, or sections. These should appear as a menu at the top of your screen. This menu should follow you around the website. It is great for navigation, and reduces the number of clicks needed to get from page to page. This is good User eXperience, and helps with Google ranking.

Beneath each category are subcategories if the main categories need to be broken down further.

Then you have the third level. These are your blog posts, your products. Notice how they’re staying beneath each category? These are called- Content Silos.

Linking internally between pages within the same silo can help build page authority, show Google how content is related and supported on your website, and hence help boost the ranking of your pages.

It also helps users navigate. They know that by clicking on a certain menu item leads them to the second level of your website, and further, in a logical manner. By the third click they are exactly where they want to be, and hopefully this means they’re interacting with your site and a transaction is made.

Why is website structure important for SEO content?

Planning your website structure and knowing where everything goes helps Google to understand it as well. If you write a blog, make sure that it is posted under the right content tree, that way you know where to find it, and so too will Google. There is a logical path down the content branch.

Secondly, knowing where your blog sits in the structure can help identify opportunities for internal linking. This helps share page authority around your website.

Thirdly, without an optimised structure, if you write on similar topics, it will dilute your SEO experience. Google will look at your website, and will assess all content as equal and won’t know what to rank first. With a structure, Google can rank your content hub post first, which users can land on and then follow internal links to other articles which are related.

A good way to think of your website and why structure is important, is to imagine your website is a house.

The front page of your website is like opening the front door to your house. You can get a good idea of the size of the house, how many bedrooms, what the kitchen is like, the lounge room and so on.

When you walk through the house you can choose a room to view for more details- poke your head into a bedroom, or sit at the dining table. This is like clicking on a menu item and going to the second tier of your website.

Page Structure for SEO

This is also called on-page SEO, these are more than just the words on-page, it is how they are set out and used to help with the SEO. The way the words are used and tagged and placed helps crawl bots index the page and identify the content on the page quicker and easier.

What is the ideal structure for a page?

There has been a traditional structure for web pages since HTML was introduced. You have the page title and heading, and then you have the body content.

It is simple and it works. It allows the web crawlers to know what each page is about (title) and then what content there is to relate to that title (body content). And these two things need to be related otherwise the crawl bots will not rank your page highly.

What do we mean by this? If the title of your page says About Us, then we would not expect to see offers and calls to action to buy a product or service. Google will not rank your site, and if someone ever made it to your ‘About Us’ and read those things, they would probably click away, and so your bounce rate would be high, which is another factor Google would consider in the overall ranking of your website.

Optimised heading structure for SEO

A heading structure for SEO content is quite intuitive. There are HEADING tags that go from HEADING 1 (H1), all the way down the hierarchy to H6. H1 is more important than H2, which is more important than H3, and so on.

You can only have one H1 tag per page, which indicates to the reader, and to the crawl bots, what the page is about.

You can have as many H2’s and H3’s and so on, but they must sit beneath each other. You can’t randomly have an H4 tag sitting there without the H2 and H3 beforehand. The crawl bots will struggle to index the content, and it will not be effectively ranked from search.

Heading Structure

Why is page structure so important for SEO Content?

Structure helps both the reader and the crawl bots.

A person reading the text can be led by the Heading tags. It breaks up the text from being just a Wall of Text, into bite-sized portions of information.

If there is something specific a reader wants from your article, and it has a heading tag, it stands out. The reader can quickly find the information they’re looking for without having to read through all the text.

Think about recipe blogs. You want the ingredients and recipe, but you need to read pages of backstory before you get there. The smarter recipe sites now have a jump button that takes you directly to the recipe.

Structure helps with SEO through the heading tags. IF you wrote a blog about widgets, and you had some H2 tags about blue widgets, and someone was searching for blue widgets, because it is in a heading tag, it will stand out. Google will know that you have a blog with a section about blue widgets and will serve this up based on the user’s search intent.

We have written about how content is important, and what kind of content you need. You still need to write it, which can be a time-consuming activity. Or you can get a copywriter to write it for you.

Copywriting is not like regular writing, like writing an email, or in your diary, or a novel. It isn’t a casual blog where you happily write about your day and include a banana bread recipe.

Copywriting is a tool, it has a function. It is a blend of function and craft.

SEO Content starts with an original idea.

This is a misnomer. As Mark Twain said- there is no such thing as a new idea.

But, you can put your spin on a current idea. Plus you need to be shown to be up on current knowledge, just like your competitors.While there may not be original ideas, you can have original SEO content. That difference is powerful.

For example, Google’s Core Web Vitals update. There are a lot of sites online with news and opinions about this update, and many of these sites have similar views, so not original. However, if you’re an expert in this field, you need to have your opinion out there, with everyone else.

Don’t waste time trying to find an original idea. Find an idea that excites you, do some research on keywords and competitors, write something more current and better than your competitor, have an offer for people who read, and start competing on those keywords.

Keep in mind content design and imagery.

Writing words is one thing. How they look on the page is another. A well-laid-out web page can influence a user’s experience in either direction.

You don’t want big walls of text. These can be hard to read and put people off. It looks intimidating and can be hard to find nuggets of information buried in the many sentences and lines of writing.
Have short, punchy paragraphs. One or two sentences and you’re done. Then have a new paragraph. This both helps digest the topic, plus the white space between paragraphs helps visually.
Dot point lists break up the flow of text and can draw a reader’s eye to the information you think needs to stand out. Dot point lists can also become Snippet Results on SERP.
Consider where photos and images are going to be sitting within the text. Are they related to a part of the story? DO they connect to the text at that point? Always caption your photos and have Alt Text for your photos.
Can you get some pull quotes to draw a reader’s eye?
Work with the web designer so the layout looks professional and clean.

Always remember content, context, and search intent.

When you’re writing content for your website, be it product copy or blog articles, you need to be aware of what you’re writing and why you’re writing it. It has to have intent, a purpose for being there. It is perfectly fine if you had an opinion you wanted to share, as long as you know that such a piece probably won’t rank for any words, unless that opinion is about something current, and you use keywords strategically.

Always be aware of the search intent you are trying to serve. If you’re answering an Informational search intent, then answer the question. Don’t waffle on with mountains of text. Answer the question at the top and then have added content afterward. This helps Google index your page for that intent and helps the user get the information they need quickly.

If you’re doing product copy, talk about that product, the features and benefits. Don’t have a big story about how Joe saved the day by using this product. That story is a blog post and should go in the blog section of your website. Reading a story about Joe was not the intent of the person landing on that particular product.

Know your audience.

Write to your audience. Imagine the people you’re writing to and use their language. If your audience is technically minded then back up your statements with facts and evidence. If your audience is young and fashionable then try and speak their language and make them feel good about buying your products and services.

Use the language of your audience but be genuine when you do. Your audience can pick a fake a mile away and will leave your website and shun your brand.

If you’re writing about computers, are you writing for IT techs or young gamers? Are you talking to gamers who want high-end gear, or families who want a home PC everyone can use? You can write a blog piece for each audience, but understand that a family is not going to be interested in the top-of-the-line Alienware laptop with VR Goggles for the latest game.

SEO Copywriting best practices

There has been a lot mentioned here, and there is more you can find on the StudioHawk website, but to give you the Cliff’s Notes:

Update your content regularly. It shows Google you are current and have more content to offer people who are searching.
Always consider keywords and how to use them wisely.
Know your audience and know your niche and write to them in their language.
Have a compelling heading and top copy so people are enticed to read.
Have a purpose for your content. Why would someone read this? What purpose is it serving for your SEO and your customer base?
Write like a human, write TO someone.
Don’t be too over-complicated. It doesn’t help with the crawl bots nor with your readers.
Write with SEO in mind.

Want to know more?

StudioHawk is Australia’s largest SEO Specialist agency, with a lot of wins on the board. We have Hawk Academy to help anyone who wants to know more about SEO get an understanding and increase their knowledge. We have a friendly Australian-based studio, and a studio in London, to help you with whatever SEO you need.

Contact one of our friendly specialists to find out more!

SEO Glossary Terms

by Harry Sanders - 19 Jul 2021

SEO Glossary Terms

Chapter 1: SEO 101

Black hat: Search engine optimization practices that violate Google’s quality guidelines.

Crawling: The process by which search engines discover your web pages.

De-indexed: Refers to a page or group of pages being removed from Google’s index.

Google My Business listing: A free listing available to local businesses.

Indexing: The storing and organizing of content found during crawling.

Intent: In the context of SEO, intent refers to what users really want from the words they typed into the search bar.

KPI: A “key performance indicator” is a measurable value that indicates how well an activity is achieving a goal.

Organic: Earned placement in search results, as opposed to paid advertisements.

Ranking: Ordering search results by relevance to the query.

SERP: Stands for “search engine results page” — the page you see after conducting a search.

Traffic: Visits to a website.

White hat: Search engine optimization practices that comply with Google’s quality guidelines.

Chapter 2: How Search Engines Work – Crawling, Indexing, and Ranking

Backlinks: Or “inbound links” are links from other websites that point to your website.

Bots: Also known as “crawlers” or “spiders,” these are what scour the Internet to find content.

Citations: Also known as a “business listing,” a citation is a web-based reference to a local business’ name, address, and phone number (NAP).

Crawl budget: The average number of pages a search engine bot will crawl on your site.

Crawler directives: Instructions to the crawler regarding what you want it to crawl and index on your site.

Google Search Console: A free program provided by Google that allows site owners to monitor how their site is doing in search.

HTML: Hypertext markup language is the language used to create web pages.

Internal links: Links on your own site that point to your other pages on the same site.

NoIndex tag: A meta tag that instructions a search engine not to index the page it’s on.

Robots.txt: Files that suggest which parts of your site search engines should and shouldn’t crawl.

Spammy tactics: Like “black hat,” spammy tactics are those that violate search engine quality guidelines.

Chapter 3: Keyword Research

Commercial investigation queries: A query in which the searcher wants to compare products to find the one that best suits them.

Informational queries: A query in which the searcher is looking for information, such as the answer to a question.

Local queries: A query in which the searcher is looking for something in a specific location, such as “coffee shops near me” or “gyms in Brooklyn.”

Long-tail keywords: Longer queries, typically those containing more than three words. Indicative of their length, they are often more specific than short-tail queries.

Navigational queries: A query in which the searcher is trying to get to a certain location, such as the Moz blog (query = “Moz blog”).

Seasonal trends: Refers to the popularity of keywords over time, such as “Halloween costumes” being most popular the week before October 31.

Transactional queries: The searcher wants to take an action, such as buy something. If keyword types sat in the marketing funnel, transactional queries would be at the bottom.

Chapter 4: On-Site Optimization

Alt text: Alternative text is the text in HTML code that describes the images on web pages.

Duplicate content: Content that is shared between domains or between multiple pages of a single domain.

Header tags: An HTML element used to designate headings on your page.

Keyword stuffing: A spammy tactic involving the overuse of important keywords and their variants in your content and links.

Link equity: The value or authority a link can pass to its destination.

Meta descriptions: HTML elements that describe the contents of the page that they’re on. Google sometimes uses these as the description line in search result snippets.

Local business schema: Structured data markup placed on a web page that helps search engines understand information about a business.

Redirection: When a URL is moved from one location to another. Most often, redirection is permanent (301 redirect).

Rel=canonical: A tag that allows site owners to tell Google which version of a web page is the original and which are the duplicates.

SSL certificate: A “Secure Sockets Layer” is used to encrypt data passed between the web server and browser of the searcher.

Title tag: An HTML element that specifies the title of a web page.

Chapter 5: Technical Optimization

AMP: Often described as “diet HTML,” accelerated mobile pages (AMP) are designed to make the viewing experience lightning fast for mobile visitors.

Browser: A web browser, like Chrome or Firefox, is software that allows you to access information on the web. When you make a request in your browser (ex: “google.com”), you’re instructing your browser to retrieve the resources necessary to render that page on your device.

Client-side & server-side rendering: Client-side and server-side rendering refer to where the code runs. Client-side means the file is executed in the browser. Server-side means the files are executed at the server and the server sends them to the browser in their fully rendered state.

Critical rendering path: The sequence of steps a browser goes through to convert HTML, CSS and JavaScript into a viewable web page.

CSS: A Cascading Style Sheet (CSS) is the code that makes a website look a certain way (ex: fonts and colors).

DNS: A Domain Name Server (DNS) allows domain names (ex: “moz.com”) to be linked to IP addresses (ex: “127.0.0.1”). DNS essentially translates domain names into IP addresses so that browsers can load the page’s resources.

DOM: The Document Object Model (DOM) is the structure of an HTML document — it defines how that document can be accessed and changed by things like JavaScript.

Domain name registrar: A company that manages the reservation of internet domain names. Example: GoDaddy.

Faceted navigation: Often used on e-commerce websites, faceted navigations offer a number of sorting and filtering options to help visitors more easily locate the URL they’re looking for out of a stack of thousands or even millions of URLs. For example, you could sort a clothing page by price: low to high, or filter the page to view only size: small.

Fetch and Render tool: A tool available in Google Search Console that allows you to see a web page how Google sees it.

File compression: The process of encoding information using fewer bits; reducing the size of the file. There are many different compression techniques.

Hreflang: A tag that indicates to Google which language the content is in. This helps Google serve the appropriate language version of your page to people searching in that language.

IP address: An internet protocol (IP) address is a string of numbers that’s unique to each specific website. We assign domain names to IP addresses because they’re easier for humans to remember (ex: “moz.com”) but the internet needs these numbers to find websites.

JSON-LD: JavaScript Object Notation for Linked Data (JSON-LD) is a format for structuring your data. For example, schema.org can be implemented in a number of different formats, JSON-LD is just one of them, but it is the format preferred by Google.

Lazy loading: A way of deferring the loading of an object until it’s needed. This method is often used to improve page speed.

Minification: To minify something means to remove as many unnecessary characters from the source code as possible without altering functionality. Whereas compression makes something smaller, minification actually removes things.

Mobile-first indexing: Google began progressively moving websites over to mobile first indexing in 2018. This change means that Google crawls and indexes your pages based on their mobile version rather than their desktop version.

Pagination: A website owner can opt to split a page into multiple parts in a sequence, similar to pages in the book. This can be especially helpful on very large pages. The hallmarks of a paginated page are the rel=”next” and rel=”prev” tags, indicating where each page falls in the greater sequence. These tags help Google understand that the pages should have consolidated link properties and that searchers should be sent to the first page in the sequence.

Programming language: Writing instructions in a way a computer can understand. For example, JavaScript is a programming language that adds dynamic (not-static) elements to a web page.

Rendering: The process of a browser turning a website’s code into a viewable page.

Render-blocking scripts: A script that forces your browser to wait to be fetched before the page can be rendered. Render-blocking scripts can add extra round trips before your browser can fully render a page.

Responsive design: Google’s preferred design pattern for mobile-friendly websites, responsive design allows the website to adapt to fit whatever device it’s being viewed on.

Rich snippet: A snippet is the title and description preview that Google and other search engines show of URLs on its results page. A “rich” snippet, therefore, is an enhanced version of the standard snippet. Some rich snippets can be encouraged by the use of structured data markup, like review markup displaying as rating stars next to those URLs in the search results.

Schema.org: Code that “wraps around” elements of your web page to provide additional information about it to the search engine. Data using schema.org is referred to as “structured” as opposed to “unstructured” — in other words, organized rather than unorganized.

Structured Data: Another way to say “organized” data (as opposed to unorganized). Schema.org is a way to structure your data, for example, by labeling it with additional information that helps the search engine understand it.

Chapter 6: Link Building & Establishing Authority

DA: Domain Authority (DA) is a Moz metric used to predict a domain’s ranking ability; best used as a comparative metric (ex: comparing a website’s DA score to that of its direct competitors).

Deindexed: When a URL, section of URLs, or an entire domain has been removed from a search engine index. This can happen for a number of reasons, such as when a website receives a manual penalty for violating Google’s quality guidelines.

Editorial links: When links are earned naturally and given out of an author’s own volition (rather than paid for or coerced), they are considered editorial.

Digital PR: This is a strategy to increase brand awareness using online techniques. Common strategies include SEO, Content Marketing and Social Media.

Follow: The default state of a link, “follow” links pass PageRank.

Google Analytics: A free (with an option to pay for upgraded features) tool that helps website owners get insight into how people are engaging with their website. Some examples of reports you can see in Google Analytics include acquisition reports that show what channels your visitors are coming from, and conversion reports that show the rate at which people are completing goals (ex: form fills) on your website.

Google search operators: Special text that can be appended to your query to further specify what types of results you’re looking for. For example, adding “site:” before a domain name can return a list of all (or many) indexed pages on said domain.

Guest blogging: Often used as a link building strategy, guest blogging involves pitching an article (or idea for an article) to a publication in the hopes that they will feature your content and allow you to include a link back to your website. Just be careful though. Large-scale guest posting campaigns with keyword-rich anchor text links are a violation of Google’s quality guidelines.

Link building: While “building” sounds like this activity involves creating links to your website yourself, link building actually describes the process of earning links to your site for the purpose of building your site’s authority in search engines.

Link exchange: Also known as reciprocal linking, link exchanges involve “you link to me and I’ll link to you” tactics. Excessive link exchanges are a violation of Google’s quality guidelines.

Link profile: A term used to describe all the inbound links to a select domain, subdomain, or URL.

Linked unstructured citations: References to a business’ complete or partial contact information on a non-directory platform (like online news, blogs, best-of lists, etc.)

NoFollow: Links marked up with rel=”nofollow” do not pass PageRank. Google encourages the use of these in some situations, like when a link has been paid for.

Page Authority: Similar to DA, Page Authority (PA) predicts an individual page’s ranking ability.

Referral Traffic: Traffic sent to a website from another website. For example, if your website is receiving visits from people clicking on your site from a link on Facebook, Google Analytics will attribute that traffic as “facebook.com / referral” in the Source/Medium report.

Resource pages: Commonly used for the purpose of link building, resource pages typically contain a list of helpful links to other websites. If your business sells email marketing software, for example, you could look up marketing intitle:”resources” and reach out to the owners of said sites to see if they would include a link to your website on their page.

Unnatural links: Google describes unnatural links as “creating links that weren’t editorially placed or vouched for by the site’s owner on a page.” This is a violation of their guidelines and could warrant a penalty against the offending website.

Chapter 7: Measuring, Prioritizing, & Executing SEO

API: An application programming interface (API) allows for the creation of applications by accessing the features or data of another service like an operating system or application.

Bounce rate: The percentage of total visits that did not result in a secondary action on your site. For example, if someone visited your home page and then left before viewing any other pages, that would be a bounced session.

Channel: The different vehicles by which you can get attention and acquire traffic, such as organic search and social media.

Click-through rate: The ratio of impressions to clicks on your URLs.

Conversion rate: The ratio of visits to conversions. Conversion rate answers how many of my website visitors are filling out my forms, calling, signing up for my newsletter, etc.

Google Analytics goals: What actions are you hoping people take on your website? Whatever your answer, you can set those up as goals in Google Analytics to track your conversion rate.

Google Tag Manager: A single hub for managing multiple website tracking codes.

Googlebot / Bingbot: How major search engines like Google and Bing crawl the web; their “crawlers” or “spiders.”

Pages per session: Also referred to as “page depth,” pages per session describes the average number of pages people view of your website in a single session.

Page speed: Page speed is made up of a number of equally important qualities, such as first contentful/meaningful paint and time to interactive.

Scroll depth: A method of tracking how far visitors are scrolling down your pages.

Search traffic: Visits sent to your websites from search engines like Google.

Time on page: The amount of time someone spent on your page before clicking to the next page. Because Google Analytics tracks time on page by when someone clicks your next page, bounced sessions will clock a time on page of 0.

UTM code: An urchin tracking module (UTM) is a simple code that you can append to the end of your URL to track additional details about the click, such as its source, medium, and campaign name.

Why Page Speed is Important for SEO

by Harry Sanders - 27 May 2021

Your Page Speed and why it is Essential to Good SEO

Page speed is about user experience. If things load slow, your users will easily click away from your website and visit your competition. Also, the slower your page loads, Google will notice and penalise you in rankings.

You need to know how to improve page load speeds, need to know what page speed optimisation is, and why it is so key to your overall SEO campaign. Read on and let us inform you of these things and more.

What is page speed?

Moz states that:

“page speed can be described in either “page load time”, (the time it takes to fully display the content on a specific page) or “time to first byte” (how long it takes for your browser to receive the first byte of information from the webserver).”

https://moz.com/learn/seo/page-speed

The time it takes to load an entire page is generally the metric used to determine load speed. This is all influenced by your server speed, your internet connection, file sizes, and image compression, to name but a few.

Pagespeed Tools we recommend

To check your page speed and find out if there is something wrong or not, we have three tools we’d recommend using.

GTMetrix

The GTMetrix Speed Tool is powered by Google Lighthouse, an open-source automated tool you can use to improve the performance of your website.

To get a full result and report from these guys you do need to pay, but you can get yourself a good idea of your page speed from the free services they provide.

One of the best features of GTMetrix is the waterfall. This shows the load order and load speed of each element on your website. Web Developers love this feature as it gets into the nitty-gritty of what bits are loading faster than others and in which order they are loading.

There seem to be very few downsides to using GTMetrix, and the ones found by reviewers online are kind of small.

They only have a server in Canada, this can give inaccurate results for faraway places like Australia.
There can be information overload and you might not be able to identify all the key data you need.
Although it finds issues, some people feel it doesn’t explain how to fix these issues.

Google Page Speed

https://developers.google.com/speed/pagespeed/insights/

This site is very comprehensive when it comes to details about your page speed. It is, however, a very hard judge on sites. Your score on the Google Page Speed may be lower than on other sites.

With Google’s Core Web Vitals coming out, page speed has suddenly become very important, so sites like Google’s Page Speed are also important. It slips right in next to the Core Web Vitals in regards to helping you rank high with SERP.

Much like GTMetrix, there are few CONS to using the Google Page Speed tool. Feedback includes, suggestions for fixing the issues can be full of technical jargon and complicated for the common web user. Also, some of the performance requirements suggested can be almost impossible to meet.

Semrush Site Audit

https://www.semrush.com/siteaudit/

Semrush has a vast array of tools to help make your website awesome, including their Site Audit tool to help with your Page Speed.

It presents data in graphs and charts and is very easy to read. The report can give you a diagnosis of any problems it finds.

While it gives you a lot of great information and presented very well, again it may take a bit of technical knowledge to fix any identified issues. There is a large knowledge database you can read, but still, it can be intimidating to the casual web user.

How Page Speed affects SEO

Page speed, as an element of SEO, has been a factor since 2010. Google upped the ante in 2018 with their SPEED update which included mobile factors in their scores.

Ranking Higher

Google ranks pages higher with a better and faster loading time. This makes it a direct ranking factor for SERP. Google measures this speed in time to the first byte.

Crawling your Site

Crawlers can crawl more pages on a site if it loads faster. This convenience to crawl and get data quicker helps lift you up the ranking.

User Experience

User Experience is a factor with page speed. If a user gets annoyed at a slow loading page, they will click away and visit your competition. This bounce rate impacts negatively upon your ranking with Google.

Ways to Increase Page Speed

Now that you’ve used one of the tools and found out your page is loading a little slowly, and you understand the importance of page speed, how do you increase it?

Increasing Mobile Page Speed

Reduce page redirects

www.page.com redirecting to m.page.com redirecting to m.page.com/home
Each of these redirects makes your page load slower.
Optimise your images
- Fix file size to fit the page appropriately.
- Use image tag attributes to fix an image rather than loading slowly.

With mobile browsing always on the increase, and it being a big part of Google’s speed update, having a fast page load for mobile users is important and smart.

Increasing Desktop Page Speed

Make sure you have good server speed.
Optimise your images
- Like above, make sure the image file size is smaller.
- And the positions are fixed.
Have a good site structure
- Optimise this so the page loads logically and quickly, with all the images and links working quickly.
Reduce Javascript and HTML
- The more 3rd party code needing to be loaded slows the whole process down.

Google’s Core Web Vitals

Core Web Vitals is a massive update to Google happening this year, 2021. There is a lot out there online regarding the ins and outs of this, but in a nutshell, there are 3 core vitals Google is focussing on in this update to help with ranking. They are:

1. Visual Stability –

Cumulative Layout Shift (CLS).

How stable is a page when it loads?
Do the images move slightly while the page loads?
Do links move while you’re clicking on them, so you click somewhere else?

This is a bad User Experience. To fix these issues, do the following:

Use set size attributes. Browsers will know exactly how big an element is and so won’t need to adjust the page to fit when the element loads.
Advertising elements have reserved space so nothing loads there and suddenly appears.
Add new elements below the fold so they don’t push stuff around when they load.

2. First Input Delay FID

This is how quickly a person can interact with your page, such as:

Choose a menu option.
Click on a link.
Enter an email address.
Open up the accordion text on a mobile website.

This factor is important because it reflects how real people use your website. For a page that is mostly passive content, FID is not that important. IF you have surveys, subscription forms, or the like, then this factor is very important.

How to fix this problem?

Minimise any javascript on your page.
Remove any non-critical 3rd party scripts as well.
Use a browser cache.

3. Large Content Paint LCP

How long does it take for your page to load from a user’s point of view? This element is directly linked to Page Speed and what we’ve been talking about.

This metric is different from other speed metrics because it is USER CENTRIC, and how a user is able to see and interact with a page.

If you find any of these issues with your website, some ways to fix them include:

Upgrade your web host.
Set up ‘lazy loading’- images only load when a user scrolls down your page.
Remove large page elements.

How will Core Web Updates affect your site?

Search intent is now being strongly coupled with user experience. The perception of what the most helpful site for users is being fine-tuned.

You may have the best content, but if your website loads slowly, and the images shift and change during loading, your ranking will suffer.

Engage with your SEO experts to run reports and find ways to hone the speed and user experience of your online presence.

Recommended Reading: Complete Guide to Technical SEO – Core Web Vitals

Page Speed Benchmark

Where should your page sit in terms of benchmark loading speed? What is the average speed of websites that you compete with?

According to Semrush’s How fast is fast enough article:

if your site loads in 5 seconds, it is faster than approximately 25% of the web
if your site loads in 2.9 seconds, it is faster than approximately 50% of the web
if your site loads in 1.7 seconds, it is faster than approximately 75% of the web
if your site loads in 0.8 seconds, it is faster than approximately 94% of the web

What ways Different CMS can Improve Page Speed?

If you’re using one of the popular Content Management Systems out there, what are they doing to help you with Page Speed in preparation for the Core Web Vitals update?

Page Speed for Shopify

In August of 2020, Shopify rolled out its Site Speed feature. There is a speed dial at the bottom of your Shopify site when you’re logged into the CMS. What this does is measure your speed in comparison to other Shopify merchants.

Shopify is fast, in general, and what you’re looking at is the speed of your site compared to others in your arena.

Google Lighthouse is used to calculate this speed score for you.

Has this score, looking low, affected sales? No, not really. Is there a reason it is so slow? Images, probably. Should you be that concerned with your Shopify site? In a nutshell, no.

Page Speed for WordPress.

Over 40% of the web is powered by WordPress. The fact that it is open source is fantastic. It means anyone with some skill can create plugins to make your WordPress experience better. The problem is, if you begin to install a lot of widgets and plugins, they can dramatically slow your website down.

Many WordPress sites are static, as in they’re just blogs or photo albums, so the general tips for speeding up the site would apply – image compression, structure, and so forth.

But for those who have plugins and widgets, consider these options:

High-performance WordPress hosting.
Check the aerodynamics of your theme.
- Disable features you don’t need or don’t use.
Mobile Responsive Theme.
- Very important with the incoming Core Web Vitals as well.
Quality plugins don’t slow your site down as much.
- Quality can cost, but it pays off in the long run.
Limit your posts on the Blog feed.
- No need for 50 thumbnails for past blogs.
Look into server-side caching.

Page Speed for a Headless CMS.

A headless CMS is a system that ONLY manages the content. A different tool manages and delivers the front-end web experience. An API fetches the content for the front page when required.

A dedicated front end can mean much faster speed results. Less code as well for the bots to read, so more speed.

The separation of these two elements also means you can update and recode the front end dynamically while the back end with the data stays static.

The Takeaway

Page speed is becoming a much more important factor in page ranking, thanks to the pending Core Web Vitals. Milliseconds matter.

As it is a reflection of User Experience, and the Mobile First approach is a big factor, the more streamlined you can make your website, the better it will serve you for Google ranking, and the better it can serve your future customers.

We have many other articles on our website to help you with your SEO, and we have our Hawk Academy to teach you how you can work SEO yourself.

If you want to know more or want some awesome people working on your SEO, contact us today and get on board for as little as $500 a month.

The Dangers of Keyword Stuffing in SEO and How to Avoid It

by Harry Sanders - 11 May 2021

Stuffing the Christmas turkey is a fun and delicious kind of stuffing. Stuffing keywords onto your website is neither fun nor delicious. Keyword stuffing is an SEO tactic from the dark ages of the internet, back at the turn of the century, which was used to game SEO results and page ranks with Google, Yahoo, and Alta Vista search engines. It is frowned upon now, and Google can even punish you if you try this Black Hat SEO practice.

Let’s help you define keyword stuffing, what it is and why it is bad for you. Then you can have some Christmas pudding.

What is Keyword Stuffing?

Keyword Stuffing is the practice of including as many SEO keywords within the page copy as possible. It is used to manipulate page value as assigned by Google and to get a page ranked higher in SERP (Search Engine Results Page).

Web pages that are jam-packed full of keywords do not read naturally. Google’s crawl bots can pick this up, heck, even the people reading your content will pick it up and react negatively towards you. More often than not they’ll click away from your website and visit that of a competitor.

Keyword Stuffing, a brief history.

The practice of Keyword Stuffing began in the late 1990’s early 2000’s, in the era of Netscape and Yahoo search engines, when Google was just a baby.

The Big Three search engines- Yahoo, Alta Vista, and Google, relied heavily on keywords to rank a website based on search results. So people would fill their pages, the Meta tags, and other places, with keywords just to gain rank. It didn’t need to make sense or read well so long as they had the keywords on the page.

Web developers used a few different techniques to do this:

Invisible Text– matching the font colour to the background colour of the website.
Block Paragraphs– just paragraphs full of keywords.
Unnatural Repeating Words– where they would repeat the keyword in the web copy.
Meta Data Stuffing– cramming keywords into the text you’d see once you got a search result.
Spamdexing – including a lot of internal links within a paragraph or set of web copy text.

When Google began updating and releasing algorithms to identify and fight keyword stuffing, SEO experts had to find a way to gain page rank in other ways.

Keyword stuffing became an unpopular practice, labeled as ‘Black Hat SEO’, and was punishable by Google.

Keyword Stuffing Examples

Invisible text

This is the practice of matching the colour codes of the text with the colour of the website. This way the text appears invisible to the reader, but the Google crawl bots can still read it for ranking purposes.

Unnatural repeating of keywords.

This is where you repeatedly use the keyword in the web copy but it just doesn’t look natural or read well.

“If you want the best car service then you need to bring your car to Mike’s Mechanic for your car service. For the month of April he has a special on car service – a 25% discount on car service if you book before the end of April. So come to Mike’s Mechanic for your car service today. The best car service in the southern suburbs”

Can you tell what the keywords we used for the above example? How awful does it sound reading that copy? Yeah, we agree.

Spamdexing

This is a kind of Keyword Stuffing that is a little different. It is the overuse of internal linking within a document, and by overuse, we’re talking 2 or 3 times in a paragraph.

It’s great if you have a lot of content on your site which you feel readers need to see, but the purpose of your blog, or webpage, is to feed your reader the information they want there. It’s not to send them off to ten different pages on your website.

Meta Data Stuffing

The couple of lines of text you see on the SERP is called Metadata. People would fill this space with keywords, again, to try and improve their SERP ranking.

Unscrupulous SEO people would also cram keywords into the alt tags of images.

Both of these locations were generally not visible to the reader, but the Google crawl bots could read and index these locations.

Is Keyword Stuffing Good For SEO?

The short answer to this question is NO.

The longer answer is, well, longer.

Keyword Stuffing doesn’t read for a human. This means people will land on the page, not like what they see, and leave. This damages your User eXperience, and hence your SERP rankings. Google will see this traffic trend of people bouncing from your website and heading somewhere else, which will devalue your page authority.

Search intent can also be damaged if the page is not what the internet user was expecting or looking for.

Google Search Central says – “filling pages with keywords or numbers results in a negative user experience, and can harm your site’s rankings.”

Penalties for Keyword Stuffing

If Google determines that you have been found to be keyword stuffing, they can demote your page or they can remove your page altogether.

Trying to get yourself back up the rankings after Google has penalised you can be a long, hard slog. Best to not do it in the first place.

Google Algorithm updates which targeted Keyword Stuffing.

Google releases algorithm updates all the time. The bigger ones target certain gaps and try to plug them. They’re trying to level the playing field for all people online. Some of these algorithm updates specifically targeted keyword stuffing.

Panda Update – Feb 24th, 2011.
- Assigned a quality score to web pages which was used for ranking and page authority.
- Checked for ‘thin content’, user-generated spam and keyword stuffing.
- Updates to this algorithm over the years have made penalties and fixes quicker.
Hummingbird – August 22nd, 2013
- Better results with search intent.
- Helped with non-exact keywords (latent semantic indexing)
- Helped Panda find and penalise Keyword Stuffing.
Bert – October 22nd, 2019
- Focused on ‘poorly written content’, which included keyword stuffing.
- Google now rewarded good writing!

How to Avoid Keyword Stuffing

There are a few ways to avoid keyword stuffing, which seem pretty self-explanatory.

Write like a human. You need to write content that sounds human and can be read by a human. Make it engaging and what the user wants to know.
Do effective Keyword Research. The better your keyword research the more keywords you will have to use, the better the search results, and the better your page will rank.
Write GOOD COPY. A pretty basic piece of advice, but if you take the time to write good quality copy then you’ll be assured that the message is strong enough that you won’t need to stuff keywords into your site.
Increase the number of words. This can give you more room when it comes to keyword density.

What is keyword density?

Keyword Density in SEO

There is a formula for understanding what keyword stuffing is and how dense your work is with keywords. It is expressed as a percentage of keywords versus the total number of words on the page.

If you include 5 keywords in a paragraph of 50 words, the keyword density will be more than if you had 5 words for a page of 200 words.

The ideal keyword density percentage is not officially known, but experts believe it to be about 2% of the copy on a page.

Want to know more about SEO? Join Hawk Academy!

If you want to learn about Black Hat SEO, such as Keyword Stuffing, and why it’s bad for you. Or if you want to know more about the art of SEO, then check out our Hawk Academy.

Learn easy SEO lessons from the Founder of StudioHawk- Harry Sanders.

If you understand that SEO is vital for the health of your website, and your business, but you don’t have the time or knowhow to fully crank your Google rankings up by using it then you need to have a chat with some of our White Hat SEO experts here at StudioHawk.

Our friendly team of experts can have you climbing the rankings of Google, with SEO practices, backlinks, and content written for you.

And remember- the best stuffing is the kind you eat or the kind you put into teddy bears.