If you’ve been following the news about Wikipedia over the past few weeks, you might have heard about ‘Knowledge Engine’, the secret search engine project that was supposedly going to take on Google.
The idea of a “transparent search engine” created by the Wikimedia Foundation has definitely been nixed, as we reported in our round-up of key SEM news stories last Friday week.
Contrary to repeated insistences by those at the top, however, there was such a project in development at one point. It served as a flashpoint in a crisis which is still taking place at the heart of Wikimedia, resulting in the resignation of its Executive Director, Lila Tretikov, last week.
But why has Wikimedia’s ‘Knowledge Engine’ been so controversial, and what bearing does the search project still have on the future of the Wikimedia Foundation?
Wikimedia in meltdown
It hasn’t been a good year for the Wikimedia Foundation – and it’s barely even March. In just a couple of months, the non-profit which hosts Wikipedia has seen numerous resignations from key staff members, a vote of no confidence in a recently-appointed Board member, and the resignation of its Executive Director.
Wikimedia has been seeing walk-outs from key staff members ever since late 2014, but things have undeniably accelerated over the past several weeks. Former Deputy Director of Wikimedia Erik Möller, who left in April 2015, described the recent situation as “very much out of control” and “a crisis.”
The Wikimedia Foundation’s internal crisis has resulted in the resignation of its Executive Director, Lila Tretikov, after less than two years in the position
Trust has been steadily eroding between Wikimedia’s community of volunteer editors and its Board of Directors, who are seen as moving too much towards a ‘Silicon-valley’-style focus on technological projects, and away from the organisation’s culture of transparency, integrity and openness.
Things came to a head with the secrecy and confusion surrounding the ‘Knowledge Engine’, a project which many of those in charge insisted was not and never had been a search engine, even when all evidence began to point to the contrary.
Jimmy Wales’ search history
Though we now know that the ‘Knowledge Engine’ project, officially called ‘Discovery’, is not going to be a search engine in its own right, it wouldn’t have been the first time that Wikipedia founder Jimmy Wales had struck out into the world of search.
He has a history of two previous search engine ventures: ‘Wikiasari’, in 2006, and ‘Wikia Search’ in 2008. Neither project lasted very long, but Wales wrote in a blog post after Wikia Search folded that, “I will return again and again in my career to search, either as an investor, a contributor, a donor, or a cheerleader.”
So it isn’t that surprising that when details about Wikimedia’s ‘Knowledge Engine’ surfaced online, people jumped to the conclusion that Wikimedia and its founder were making another “return to search.”
Not least because in a September 2015 announcement, the Knight Foundation, which provided a grant of $250,000 to fund Wikimedia’s ‘Knowledge Engine’, described the project as “a system for discovering reliable and trustworthy public information on the Internet.”
Now, that might not necessarily mean a search engine, but it does sound a lot like one, and Wikimedia wasn’t awfully forthcoming with information to clarify what the project was, if not a search engine.
On 11th February, Jimmy Wales publicly refuted the notion that the Knight Foundation grant was “in any way related to or suggestive of a google-like search engine”, adding that, “no one in top positions has proposed or is proposing that WMF should get into the general ‘searching’… It’s a total lie.”
But that same day, Wikimedia Foundation published the original Knight Foundation grant agreement, containing its own description of the project as “the Internet’s first transparent search engine”. It also draws contrasts between Wikipedia’s approach to knowledge access and that of “commercial search engines”, setting out the intent to “create an open data engine that’s completely free of commercial interests.”
A series of conceptual designs for the Wikipedia front page, from a Discovery presentation in November 2015, greatly resemble a search engine in layout and focus.
By WMoran (WMF) [CC BY-SA 3.0], via Wikimedia Commons
On top of this, the Signpost – Wikipedia’s own community-written and edited newspaper – revealed details of three internal documents which had been leaked to the paper but not made public.
Dating back to April 2015, the documents delve deeper into the priorities of commercial search engines and how these skew the information which is presented to users, contrasting them with a “Wikipedia Search” which would be “Private and secure” with “Transparent results rankings”, “Locally relevant information” and “Global representation in all languages and cultures.”
They also feature mock-ups of what a Wikipedia search engine could look like, drawing results from across Wikimedia’s projects as well as external sources like Fox News.
The case for Wikipedia Search
You might be thinking, so what if Wikimedia does want to go into search? And there are definitely reasons why it would be an attractive idea. Wikipedia has seen a lot of traffic dropping off recently as Google instead displays key information from its articles directly on the results page.
Wikipedia is still the authoritative source of information, but it’s being deprived of visitors to the encyclopaedia itself, making it even less likely that people will click through to other articles on the site, make edits to them, or see calls for donations from one of its funding drives.
So if Wikipedia is going to dominate the top results for any given Google search, why not create a search engine which sends users straight there? And it’s not just about Wikipedia, either – Wikimedia presides over a diverse range of knowledge projects covering e-books, travel, science, data and more. Any search system which properly interlinks its different projects would be a source to be reckoned with.
The grant agreement with the Knight Foundation poses the question, “Would users go to Wikipedia if it were an open channel beyond an encyclopedia?” By making Wikipedia (or Wikipedia’s ‘Knowledge Engine’) a one-stop-shop for the world’s knowledge, along with other incentives like ad-free and transparent search results, Wikipedia might have been able to reclaim the traffic it has been losing to Google and redirect it across its own various sites.
A mock-up of a Wikipedia-style Search Engine shows how information could be drawn from across Wikimedia’s various knowledge projects to provide context on current events.
By Wikimedia Foundation [CC BY-SA 3.0], via Wikimedia Commons
The notion of a ‘Wikipedia search engine’ is problematic for much of its community, however, who are concerned that it would represent a move away from human-curated and authored knowledge towards automatically generated content.
It would also have been an extremely ambitious project, costed at around $2.5 million in its initial stage and possibly running into the tens of millions, diverting attention and funds away from other areas of Wikimedia which sorely needed them. And, as we’ve established, the way in which the project was being implemented seemed to run directly counter to Wikimedia’s key principles of openness and transparency.
Farewell to search?
The question of how a genuinely transparent and non-commercial search engine would impact the world of search is a fascinating one, but unfortunately, isn’t going to come from Wikimedia.
While there’s no question that the Wikimedia Foundation was looking into a search engine at one point, those involved in the Discovery project have (eventually) been quite unequivocal about where its efforts are now going.
First to step forward was Max Semenik, a Discovery engineer, who stated that,
“Yes, there were plans of making an internet search engine. I don’t understand why we’re still trying to avoid giving a direct answer about it.
There has never been any actual technical work on this project. The whole project didn’t live long and was ditched soon after the Search team was created, after FY15/16 budget was finalized, and it did not have the money allocated for such work (umm, was it in April? in such case, this should have been soon after the leaked document was created).
I don’t think anybody but the certain champion of the project has considered competing with Google with any degree of seriousness.
No, we’re really not working on internet search engine. And will not work in the future. For shizzle.”
Minutes were also published from a meeting of the Discovery team with Wikimedia’s Executive Director and other key players, to discuss the fallout from the ‘Knowledge Engine’ debacle. User Experience Engineer Julien Girault had this to say about Wikimedia’s search plans:
“Three of our goals are about search. The Knight grant talks about a search engine, and some mock-ups look like google. There are legitimate reasons people might think we might be planning to create a google competitor.
People can come up with all kinds of ideas. People who understand what we are trying to do tend to like it. Surface more content. Make it easier to find content. Improve the portal. We shouldn’t be ashamed of that.
We already have a search engine. Google doesn’t care. The portal can be a great marketing tool for our projects, to help people discover all that we host. Wikispecies, wikiquote, etc.”
Director of Discovery Tomasz Finc added, “We should be clear that we are building an internal search engine, and we are not building a broad one.”
Ever since details of the ‘Knowledge Engine’ first began to surface, Wikimedia’s key players have been very keen to reiterate that they are not, repeat not, trying to compete with Google. The Wikimedia Discovery FAQ even features this line,
“Nobody at the WMF intended for the search function at WMF sites to do things to help you find things to buy, as Google does, and there is no evidence that the WMF wanted to do all the many, many things that Google/Alphabet does.”
Okay, but the Wikimedia Foundation doesn’t have to go into every single area that Google covers in order to be a competitor. I would venture that Wikimedia is still in a very real battle with Google for the attention and clicks of users who are seeking information online.
Wikipedia already features instructions on how to set your default browser search engine to Wikipedia, adding that you can reach all twelve of Wikipedia’s sister projects in the same fashion.
Some browsers already support Wikipedia’s search engine plugin by default, and with a more powerful, intelligent and comprehensive internal search engine on offer, a lot more users might opt to do this.
Improvements to Wikipedia’s internal search engine might entice more users to add it as their default search engine.
Even if Discovery is not now involved in building a search engine, its goal in improving internal search across Wikimedia’s projects is to make them a more attractive destination for users with search queries of all kinds, and keep users from moving away from Wikimedia’s various outlets – just as Google is aiming to do.
It will be interesting to see what direction the Discovery project takes following the resignation of Lila Tretikov, and what its investigation will reveal about the search needs and habits of Wikimedia users.
But it seems clear that the Wikimedia Foundation does intend to put search, in whatever form, at the centre of its future development.
The article Everything you need to know about Wikimedia’s ‘Knowledge Engine’ so far was first seen from https://searchenginewatch.com
No comments:
Post a Comment