The sad state of BBC search

I consider myself a pretty good internet searcher. If it's out there I can usually find it one way or another. However, there's one site in particular that always defeats me. I know the information is there, it's got a great reputation, it's jam packed full of newsy goodness, and it's also very poor. The BBC, for a world renowned news and information organisation is woeful when it comes to search. At the top of every page we've got a search option:

That's good as far as it goes, but it doesn't go very far. No advanced search option. No 'search help' option, no suggestions on what I can search for or how I can search for it. The BBC has highly complicated data sets - we've got archive type information going back years, news reports, current news, content from television and radio programmes, information that's available for a short period of time via iplayer, photographs and a wealth of other content. None of that is reflected in the search box that we're all shown.

Running a search really shows the limitations of their system. As you can see from the above illustration I wanted to find out more about the little boy who had been kidnapped, and had apparently been released. I looked on the BBC home page and there was no information at all, which surprised me, given that it's a new twist in the event, so I went looking for it. 'Pakistan kidnap' probably isn't the best search I could have done - perhaps adding in 'boy' or 'child' would have helped, but let's see what we got.

The first news item is 6 days old. 6 days! The next is 13 days. Further down the page I'm getting information that's dated 2007, 2004, and my favourite is 1st April 1998. For a 'news' site that's worse than bad, it's virtually criminal. The tragic thing is that the information that I want is actually on the page, but cunningly hidden away on the BBC iPlayer section to the right under 5 live breakfast. However, if and when iPlayer is updated to the next day, that information is essentially lost. What you can't see however is that further down the same page, in the 'news and sports clips' is this:

This states, quite categorically, that he's not been released. Moreover, it states that reports that he has are unfounded. As a searcher, what am I to do? Of course, looking at the dates is helpful, but is everyone going to do that? I have my doubts. However, I now have the childs name, and a second search does bring up much better information, and this time in the 'news and sports' section there's a useful news story I can listen to. As an aside, why on earth have news and sports information so closely linked? It's just not helpful. If Pakistan had been playing in a cricket match, which story would have got top billing?

BBC search has clearly never heard about Boolean operators. A search for the child's name and -kidnap returns exactly the same results. NOT kidnap is the same as well.

Let's look at another example. I was invited onto a couple of radio shows recently to talk about the sad case of the child murdered by a man she met via Facebook. I recall vaguely that the Government was introducing some initiative late last year to make children safer online, and it was widely reported on the BBC. I wanted to find out more about this, so turned to the BBC. I thought my search for "government guidelines internet child safety" was pretty good until the first result that turned up was "News - Technology - Net Industry must fight paedophiles" dated 6th January 2003. The nice little content boxes at the side for BBC iPlayer, News and Sport, Elsewhere on the web all showed no content. Really? A subject that is right in the middle of the news, with the Government talking about what can and should be done, and what they've done in the past - nothing recent at all?  Let's look at the dates on the results I do have - 4 Feb 2003, 16 Oct 2007, 6 Jan 2003, 18 Nov 2008, 6 Jan 2003, Undated (Don't ask me what's going on there!), 11 Jan 2002, 13 Oct 2000. So the most recent thing I can find is 16 months ago.

This wouldn't be so bad if I could re-rank my search based on date, but I can't. I'm stuck with what the BBC gives me - which in this case is outdated, useless information. You may say 'well, add in 2009, since you know the Government initiative started then'. I'm way ahead of you my wiley friend - I tried that, and the first result that came up with was "News - Technology - Net Industry must fight paedophiles" dated 6th January 2003. Yes - the very same result. In the end, I couldn't find what I wanted, so tried other resources instead, at which point I found what I was after (the 'zip it, block it, bin it' campaign) quickly.

One more example - there has recently, very recently in fact, been a cyclone in Fiji. I wanted to see how the BBC handled that. I threw in a slight curved ball at this point and tried my first search as 'Figi'. I got results as well, worryingly. 5 of them, all talking about the country of Figi. Most of these were however undated. Quite how that is useful to anyone, I'm not entirely sure. Also, no 'did you mean?' suggestion either. So I went back and did the search properly. This time I got a three panel approach - it's something new the BBC is trying. Columns and boxes for News, Sport, About, Weather, IPlayer, Blogs, Knowledge and Around the Web. This time I did get some information about the cyclone, and it was 10 hours old. There were also links to other news stories, and some video as well. It's a new search system that the BBC is trying out, and it's better than the previous version. If anyone from the BBC responds to this - which I doubt - they'll try and use this to counter my criticisms. Before they so however, I would suggest that they take a look at other news sites - Silobreaker comes to mind. When I ran my search there, the engine immediately gave me a dozen suggestions to focus my search. First news result was dated 46 minutes ago. I was then presented with other news stories, tweets, 'In focus' pieces, content volume by date, networks, hotspots, blogs, audio visual, quotes and 'more content'. THAT is how you run a news resource. 

Trying an external resource to search the BBC doesn't work either. Limiting both Google and Bing to the BBC site and running my searches again simply returns the same tired results. It's only when I go into the Google options and choose 'last 24 hours' do I find the information that I was after. This should not be necessary. Bing wasn't able to be any better either.

The BBC has great resources - why can't I use them? Why can't I limit a search and focus using the most basic of search operators, things that have been around for 15 years or more? Why can't I just get content in a specific form? Why can't I re-rank on date, which when you're dealing with a news site is the bare minimum you want, given the state of their search results. Why isn't there a proper search help function installed? Finally, and let's echo this loudly 'WHY IS BBC SEARCH SO BAD?' If you need to find news, don't bother with the BBC; they simply have no clue.

Comments

[Eli Bastin]

I'm rather confused by news that the BBC spends a huge amount of money on its website, meaning that services such as Six Music etc may be cut, while the search is appalling. Why do they even bother to have a search box if it doesn't 'work'? It really is quite extraordinary!

[Ruth O'Leary]

And given the huge cuts to the website budget the BBC are proposing, it's not likely to get any better any time soon.

[Nina Chachu]

Thanks for saying what I have been thinking. In fact I have basically just given up on trying to find anything via their search, which is pretty awful, as you said. And this is a website which I probably visit at least once a day! Apart from listening on the radio, watching on TV, and listening to podcasts!

[Joy]

Is it me or has the search got worse over time. The listing links certainly have. I know I can never find anything so search the site through Google, and always find it that way. Crazy.

[Karen]

Whilst you can't do this from the first results page, you can sort by date once you've restricted the results to news & sport http://search.bbc.co.uk/search?go=homepage&Search=Search&tab=ns&q=pakistan%20kidnap&order=date&scope=all

[Matthew McDonnell]

I am in charge of the BBC Search and am happy to respond to your criticisms.

One of the hardest things about creating a search for the BBC is that the corporation produces so many different types of content (news, programmes, clips, recipes, fact files, games and so on) and they need to be handled differently in search results. In the past we have tried to order everything using a relevance calculation which leads to the situation you describe where an old news story can beat a newer one because it is deemed more relevant to the query. Clearly not an acceptable outcome!

As you say, we are rolling out a new search system (e.g. www.bbc.co.uk/search/david_cameron) and this is one of the things we are addressing. The results are split up into sections and each section is ordered in the best way - so news is in date order but features, cooking recipes etc are in relevance order. (Incidentally we have split news and sport in the new system as you suggest).

You can read much more about our approach here - http://www.bbc.co.uk/blogs/bbcinternet/2010/01/bbc_launches_enhanced_search.html?page=0

We are still playing with the layout and functionality of the new search so any feedback is most welcome - there is a feedback form on the new search results pages.

There are other features in the pipeline that I hope will help things including the ability to set advanced filters on search results (limiting by section, date etc) and auto-suggest/ spelling suggestions.

We hope to have the full feature-set rolled out by the summer.

Matt

[John Hesling]

Your search was done after the start of the BBC enhancements mentioned in BBC Matt McDonnell's post above.

Previously BBC search results were even worse! and if I recall correctly at one stage BBC search was more of a google search of www. Hopefully things are improving with BBC promising further enhancements, so your concerns about a lack of advanced search features are being addressed.

Once you have found something on the BBC site where do you go from there, have you noted the controversy regarding http://www.bbc.co.uk/blogs/theeditors/2010/03/bbc_news_linking_policy.html how BBC news should link to external sources.