Last week, our friend Charlie Hull from OpenSource Connections – the experts when it comes to Search Optimization – answered one of the key questions when it comes to Search Optimization; what does it mean if my search is ‘broken’? (If you missed that post, check it out here!)
Today, we are tackling the next problem – How do you fix a broken search?
I’m Charlie Hull of OpenSource Connections and I’ve been helping companies build better search engines for twenty years. I’m going to help you find out if your search engine is broken – and perhaps help you fix it!
In my previous blog I wrote about common causes of broken search, from low quality content to lack of control of the search engine. Now we understand a little more about what might be at fault, let’s consider some ways we can fix search.
You may be able to put some of these ideas into practice immediately and some may take a little longer – but it’s important to remember that optimising your search is an ongoing process. Your content, user behaviour and wider market conditions will all change over time so you will have to keep up! Remember that measure and repeat should be your approach to optimising search.
- Finding the patterns
The best place to start is with some of your most common queries. Fix these, and you’ve made a lot of your users happy. Take a look at the 100 most common queries (you should be logging all queries entered into your search box by default):
- Can you notice any patterns?
- Are there a lot of location-based queries for example, or queries that map easily to categories of products?
- Are there queries that are best answered by a FAQ or other customer service page?
- Are there queries in a different language to what you might expect?
- Are your customers using different words than you might expect (‘pants’ instead of ‘trousers’ for example)?
- How many of these queries give zero or fewer than expected results?
- Prioritise by value
Classifying the problems you see from your query logs (and other problems with search you know about, like search speed) will give you a list of things you could fix – but you may not have time or budget to fix everything.
The next step is to assign a priority based on the value of each type of problem to your business.
If you need your users to find your office or store, location-based queries may be important. If you want them to buy things from your website, directing them to product categories may help. If jobseekers can’t find how to submit their resumé to your site, that won’t help you find them a position.
Working with your search team, agree the highest priority issues (and regularly meet to revise this list).
- Quick fixes
Here’s a short list of things you should be able to implement quickly based on your findings above.
- Manual redirects
Most search engines will let you add a fixed redirect (also known as a ‘best bet’) for particular queries. These should only be used when you know the best result for a query is a particular result or webpage – e.g. “customer complaint number” or “store in New York”. You can show a link to the ‘best bet’ at the top of the result list, or even redirect to the page itself.
This technique can also be used when the query is too generic or vague for you to give a good result, in which case it could be redirected to a category page presenting some further choices.
A good example is ‘screwdriver’ when your website sells 500 different types of screwdriver – just presenting a list of those 500 isn’t particularly helpful, but a category page showing the 12 basic types of screwdriver (‘power’, ‘Phillips’, ‘flathead’, ‘screwdriver set’ etc.) helps the user give you some more information.
Make sure you limit how many manual redirects you use – if the list gets too long, or you don’t clearly record the reason for using each redirect you may end up with unexpected side effects, or even forget why the redirect was added in the first place.
- Help with zero results
We tested this in my first blog by typing in a random stream of characters to your search box. If the user is presented with zero results and no help about what to do next they may leave your site. At the very least give them some helpful hints:
“We’re sorry but we didn’t find any results for that query. Try typing in some different words to describe what you’re looking for and we’ll do our best to help!”
Or show them a list of categories as a better starting point.
If you haven’t already implemented automatic spelling suggestion (sometimes known as a ‘Did you mean?’ feature) then this is also a great way of helping the user:
“We found no results for ‘shoos’ – did you mean ‘shoes?’
Or correct it automatically, as Google does:
“Showing results for ‘shoes’”
(search instead for ‘shoos’)
- Fix the data
Remember that even the best search engine you can buy or download won’t be able to give your users good results if your data is of low quality. If there’s a misspelled word in one item in the source data and the user types in the correct word, that item won’t appear in search results.
If whoever created the data didn’t label a record correctly then it may not appear where you think it should.
Some of the problems you identified above may be fixed easily in the source data. However, in some cases the data is fine, but your users just aren’t using the same words as you are! Perhaps you need to train your content team or engage some partners to help with labelling data.
Synonyms are a great way to translate between the terms your users type into the search box and the terms that describe items on your site. These can be very different!
Your content may have been written by specialists who are very familiar with the subject matter – your users probably won’t be as expert. An engineer might describe a ‘phillips stubby screwdriver’ and a home improvement enthusiast a ‘short crosshead screwdriver’ – if there’s no way to map between these two ways to describe the same thing then your users won’t get good results for their query.
There are various ways to implement synonyms – one way, bidirectional, phrases – and which you can use will depend on your search technology and the problems you’re trying to solve. Again, you should make sure you keep records of which problem a synonym was added to fix.
- Looking ahead
Once you’ve prioritized your search issues and tried some of the quick fixes above, you should think about how to further improve search. There’s a lot more queries than those first 100 or so of course, and even the low-volume queries can tell you something (this is often where misspelled queries appear).
You will probably also have realised that if you can set up an effective search testing process (both offline and online) you may be able to prevent some future problems – and you’ll certainly need this process to verify your fixes are working correctly and haven’t caused side effects.
You should also consider the structure of your search team. Remember that search optimization is a team sport, and the best search teams work across the business, not just the IT department. Specialist training is a great way to build search skills within the business.
Addressing these areas will help you reduce the risks of underperforming search – a topic I’ll cover in my next post.