Ajax against SEO and Web Accessibility

Ajax has become a new hot terms in web development world, since the rapid growth popularity of some pioneer ajax based web application like Gmail and Google Maps. Lots of web developers, web designers and internet users very excited with the more possibilities which can be done by ajax, many websites developed using the advantages of ajax technique, and since then lots of ready to use ajax toolkit/framework available for developers to simplify the experience of ajax integration eg: jquery, prototype, mootools, GWT, Dojo etc.

But there also some cons in using ajax, and one of the hottest is ajax against SEO & Web Accessibility, Just as ajax, SEO (search engine optimization) also been playing more and more role in web development world, SEO do the self marketing of website through search engines, while ajax providing more interactivity between the user and the web page. Untill now Search Engines Robots do not parse javascript means Search Engines wont crawled content delivered by ajax, web accessibility? since ajax rely on javascript it will break a fall on the screen readers which did not support javascript or javascript has been disabled on it.

Then how to mix ‘em? well i am not an expertise in SEO (Search Engine Optimization) neither does web accessibility, but as much as i read about SEO and web accessibility, it is not as complicated as some said about it. Here i will show you how to create accessible and SEO friendly ajax using Jquery, since i am more familiar with it rather than other toolkits, but ofcourse this technique can be used with other javascript frameworks as well.

• Create the wholepage as usual using CSS + xHTML. Web standards compatible? well thats not a law, just recommendations, but its good to follow em
• Optimize your pages as much as you feel it good
• !important Add filtering in your server side codes where the page is rendered to identify the request from normal HTTP request or via ajax by passing a parameter through ajax request, here i assumed the parameter name is ajax, so whenever var ajax has value then render only any content that needed without any html, head, body tags, else render the full page included those neccessary html tags

//@ PHP
if(!empty($_POST['ajax'])) {
	//@ Render only neccessary content
} else {
	//@ Render the whole page
}

'@ ASP
if Len(Request.Form("ajax"))>0 then
	'@ Render only neccessary content
else
	'@ Render the whole page
end if

Assumed we have page structures as follow (you can use apache rewrite or IIS rewrite module to make the links more SEO friendly):

	<div id="nav">
	<ul>
		<li><a href="" title="Welcome Home">home</a></li>
		<li><a href="" title="About Me">about</a></li>
		<li><a href="" title="My Services">services</a></li>
	</ul>
	</div>
	<div id="wrapper">
	<!-- CONTENT GOES HERE -->
	</div>

Search Engine Bots will not have a problem to crawl ‘em and they will follow and probably crawl the page based on the “href” attribute value.

Now we add the ajax functionality on each links. Download latest jquery library or used Google ajax library API, then create a .js file, name it ajax.js, then add these codes between the head tags

<script type="text/javascript" src="path/to/jquery.js"></script>
<script type="text/javascript" src="path/to/ajax.js"></script>

Modify the ajax.js like follow:

$(function(){
	var LoadMsg = 'Please Wait ...';
	$('#nav ul li a').click(function(){
    	if(!$(this).hasClass("current")) {
		var _Href = $(this).attr('href');
		$('<div id="loading">'+LoadMsg+'</div>').appendTo('body').fadeIn('slow',function(){
			$.ajax({
				type:	'POST',
				url:	_Href,
				data:	"ajax=1",
				dataType:	'html',
				timeout:	5000,
				success: function(d,s){
						$('#loading').fadeOut('slow',function(){
							$(this).remove();
							$('#wrapper').slideUp('slow',function(){
									$(this).html(d).slideDown('slow');
								});
							});
						},
				error: function(o,s,e){
							window.location = _Href;
						}
			});
		});
        }
		return false;
	});
});

Just that , now test your pages, try to disable/enable javascript and see how it works.

• Search engine bots will crawled your content as normal, because each link has written in semantic html format
• Any screen readers which has limitation in parsing javascript also will able to load your content
• What if javascript has enabled and the ajax request return error? well take a look at the ajax error handling, we simply tell the javascript to redirect the page using normal HTTP request

Do you want to see an online demo page? well here it is.

Update

My web pages were built using fully HTML, how can i adopt this approach?

If your web pages were built using fully HTML with no server side codes (ASP, PHP etc.), to adopt this approach you need to create double pages for each of them. 1 page designed with all neccessary HTML tags as a valid HTML page, including CSS and Javascript, and the other one contains only neccessary content.

. webroot/
... index.html
... about.html
... services.html
... style.css
... jquery.js
... ajax.js
... pages/
... ... index.html
... ... about.html
... ... services.html

And those pages in the root directory will be reached within these conditions:

1. First visited page either from direct access or referer link including from search engine result
2. Search Engine Bot Agent
3. Screen readers with no javascript enabled
4. ajax load failed or timeout

the structure could be looked like this:

<html>
	<head>
		<title>Page Title</title>
        <meta name="keywords" content="any keywords" />
        <meta name="description" content="any descriptions" />
		<script type="text/javascript" src="jquery.js"></script>
		<script type="text/javascript" src="ajax.js"></script>
        <link href="style.css" rel="stylesheet" media="all" />
	</head>
	<body>
		<div id="nav">
			<ul>
				<li><a href="index.html" class="current">Home</a></li>
				<li><a href="services.html">Services</a></li>
				<li><a href="about.html">About me</a></li>
			</ul>
		</div>
		<div id="wrapper">
        	<!-- CONTENT TEXT GOES HERE! -->
        </div>
		<div id="footer">
        	<!-- FOOTER TEXT GOES HERE! -->
        </div>
	</body>
</html>

Web pages inside /pages/ will only accessed via ajax in screen readers with javascript enabled. It only contain Informational data.

Modify ajax.js and replace the content with these codes:

$(function(){
	var LoadMsg = 'Please Wait ...';
	var AjaxPath = 'pages/';
	$('#nav ul li a').click(function(){
		if(!$(this).hasClass("current")) {
			var _Href = $(this).attr('href');
			$('#nav ul li a').removeClass("current");
			$(this).addClass("current");
			$('<div id="loading">'+LoadMsg+'</div>').appendTo('body').fadeIn('slow',function(){
				$.ajax({
					type:	'GET',
					url:	AjaxPath+_Href,
					dataType:	'html',
					timeout:	5000,
					success: function(d,s){
							$('#loading').fadeOut('slow',function(){
								$(this).remove();
								$('#wrapper').slideUp('slow',function(){
										$(this).html(d).slideDown('slow');
									});
								});
							},
					error: function(o,s,e){
								window.location=_Href;
							}
				});
			});
		}
		return false;
	});
});

!important protect files inside /pages/ directory from being indexed by search engine using robots.txt

Sample of implementation here