Skip to the content

MoD delves into open source 'deep web' mining



Ministry plans hackathon as a step towards digging into the deeper reaches of the World Wide Web

The Ministry of Defence (MoD) is planning a weekend hackathon as part of an effort to sharpen up its web mining operations.

It is aimed at developing new approaches to web searches that could help government to pull in information relevant to different projects from a wider range of sources.

The MoD is running the event, named MoDHack, in London 25-27 September along with the government’s Defence Science and Technology Laboratory and the innovation support body Knowledge Transfer Network.

It is part of the Catalysing Defence Innovation through Science and Technology project, in which the department is looking at new ways of working with non-traditional defence suppliers.

The hackathon is open to software developers, data scientists and innovators who want to use open source tools to mine the ‘deep web’, the part of the World Wide Web that is beyond the reach of standard search engines. This can include web pages that are password protected or on encrypted networks.

“Today’s web searches use a centralised, one-size-fits-all approach that searches the Internet with the same set of tools for all queries,” the MoD said in announcing the event. “While that model has been wildly successful commercially, it does not work well for many government cases.”

New methods

The aim will be to create new methods for users to organise and search subsets of information that would be difficult to identify through ordinary web searches. This can involve the creation of domain-specific indexing and search paradigms to improve the discovery of content and information extraction.

As part of the effort, the MoD is promoting the use of open source tools developed by the US Defense Advanced Research Project Agency.

At the hackathon it will judge the proof of concepts for different projects and provide a prize – a day out with the Royal Navy Underwater Escape Training Unit – for the overall winner.

Anyone interested can find more details at at the EventBrite link.

Image By Fir0002 CC-BY-SA-3.0 through Wikimedia



Register For Alerts

Keep informed - Get the latest news about the use of technology, digital & data for the public good in your inbox from UKAuthority.