Microsoft To Open Source A Key Piece Of Its Search Engine Bing
by Laurie Sullivan@lauriesullivan, September 7, 2016
Microsoft engineers are working to open source key parts of its search engine Bing.
There are three important pieces listed on the developers’ Web site GitHub: BitFunnel, NativeJIT, and Workbench. All three work together, but have separate functions.
One media outlet reports that Microsoft engineers have begun to provide code and details of the BitFunnel full-text system, which the company describes as a library for high-performance full-text search over “a chunk of the Internet, spread across thousands of machines.”
BitFunnel is based on an algorithm that identifies and ranks documents according to queries linking to keywords, phrases, and mathematical expressions.
The few open-source projects that are so far available are minimal, but one on the way from Microsoft’s Bing division should have applications beyond search, including a framework for generating just-in-time compiled machine code.
For advertisers this means familiarity in other Microsoft applications that will likely include some form of search function.
“BitFunnel is but a small cog in the massive machine that is Bing and today there is no way to boot up the system outside of our data centers,” writes Michael Hopcroft, manager for Visual Studio Languages at Microsoft, wrote in a blog post.
Hopcroft, who has spent six years developing the cloud-scale infrastructure for the Bing search engine, explains the “biggest challenge in making BitFunnel relevant to the open source community will be removing Bing-specific functionality that has deep dependencies on the internals of the rest of the Bing web crawling and index serving infrastructure.”
The developers Web site GitHub points to three projects: BitFunnel, NativeJIT, and Workbench. All three work together but have separate functions.
MediaPost.com: Search Marketing Daily
(6)
