Predicting the text in redacted documents is close to reality

Releasing delicate information with big black bars all over it has kept secrets safe for years - but not for much longer, maybe.

For those with secrets they want to keep, redacting documents is a pretty important thing to get right. It’s necessary to understand how to redact documents, firstly - look to Southwark Council, which in February uploaded its controversial agreement with developer Lend Lease for the regeneration of the Heygate Estate in a form that let people copy and paste the text underneath the black bars.

But it’s also necessary to know which parts of a document to redact so that the context from the stuff left open doesn’t give the game away. There is always, however, information left behind. The choices made in how to block text - be it with other bits of paper, or black marker pen, or even by typing out new words and then covering those up - can reveal something about the person doing the redacting. Different agencies had different redaction standards at different times, which gives a further clue as to what technique is needed. Each typeface fits into the space under a bar in a limited number of contextually-relevant ways, as well.

In the New Yorker, William Brennan reports on The Declassification Engine, an intriguing attempt by a group of academics to use these clues to try and crack any redacted text. A snippet:

Together with a group of historians, computer scientists, and statisticians, [Columbia history professor Matthew] Connelly is developing an ambitious project called the Declassification Engine, which, among other things, employs machine-learning and natural language processing to study the semantic patterns in declassified text. The project’s goals range from compiling the largest digitized archive of declassified documents in the world to plotting the declassified geographical metadata of over a million State Department cables on an interactive global map, which the researchers hope will afford them new insight into the workings of government secrecy. Though the Declassification Engine is in its early stages, Connelly told me that the project has “gotten to the point where we can see it might be possible to predict content of redacted text. But we haven’t yet made a decision as to whether we want to do that or not.”

One of the things that jumps out in here is the parallel between the "mosaic theory" - where "pieces of banal, declassified information, when pieced together, might provide a knowledgeable reader with enough emergent detail to uncover the information that remains classified" - and critics of the NSA who realise that mass collection of metadata rather than the actual data of communications is, in many ways, just as bad.

Redacted Iraq War info at a 2004 US Senate press conference (Photo: Getty)

Ian Steadman is a staff science and technology writer at the New Statesman. He is on Twitter as @iansteadman.

Yu Ji/University of Cambridge NanoPhotonics
Show Hide image

Nanoengine evolution: researchers have built the world’s smallest machine

The engine could form the basis of futuristic tiny robots with real-world applications.

Richard P Feynman, winner of the Nobel Prize in Physics in 1965, once remarked in a now-seminal lecture that a time would come where we would “swallow the doctor”. What he meant, of course, was the actualisation of a science-fiction dream – not one in which a universal cure-all prescriptive drug would be available, but one in which society would flourish through the uses of tiny devices, or more specifically, nanotechnology. 

First, a quick primer on the field is necessary. Nanoscience involves the study and application of technologies at an extremely tiny scale. How tiny, you ask? Given that one nanometre is a billionth of a metre, the scale of work taking place in the field is atomic in nature, far beyond the observational powers of the naked human eye.

Techno-optimists have long promoted potential uses of nano-sized objects, promising increases in efficiency and capabilities of processes across the board as a result. The quintessential “swallow the doctor” example is one which suggests that the fully-realised potential of nanotechnology could be applied to medicine. The idea is that nanobots could circulate our bodily systems in order to reverse-engineer the vast array of health problems that threaten us.

It’s natural to be sceptical of such wild aspirations from a relatively young field of study (nanoscience unofficially began in 1959 following Feynman’s lecture “There’s Plenty of Room at the Bottom”), but associated research seems to be gaining widespread endorsement among prominent scientists and enthusiasts. Ray Kurzweil, Director of Engineering at Google, thinks a booming nanotechnology industry is crucial in the creation of a technological singularity, while futurist and viral video philosopher Jason Silva believes the technology will help us cure ageing.

The high-profile intrigue surrounding nanotechnology means that word of any significant developments is certain to stimulate heightened interest – which is why researchers’ achievement in building the world’s tiniest engine this month is so significant.

Reporting their results in the journal Proceedings of the National Academy of Sciences, the University of Cambridge researchers explained how the nanoengine was formed and why it represented a key step forward in the transition of the technology from theory to practice.

The prototype nanoengine is essentially composed of charged particles of gold, bound by polymers responsive to temperature in the form of a gel. The engine is then exposed to a laser which beams and heats the device, causing it to expel all water from the polymeric gel. The consequence of this is a collapsing of the gold particles into an amalgamated, tightened cluster. Following a period of cooling, the polymer then begins to reabsorb the water molecules it lost in the heating process, resulting in a spring-like expansion that pushes apart the gold particles from their clustered state.

"It's like an explosion," said Dr Tao Ding from Cambridge's Cavendish Laboratory. "We have hundreds of gold balls flying apart in a millionth of a second when water molecules inflate the polymers around them."

The process involved takes advantage of the phenomenon of Van der Waals forces – the attraction between atoms and molecules. The energy from these forces is converted into elastic energy, which in turn is rapidly released from the polymer. "The whole process is like a nano-spring," said Professor Jeremy Baumberg, who led the research.

Scientists have been tirelessly working towards the creation of a functional nanomachine – one which can effortlessly swim through water, gauge its surroundings and communicate. Prior to the research, there was a difficulty in generating powerful forces at a nanometre scale. These newly devised engines, however, generate forces far larger than any previously produced.

They have been named “ANTs”, or actuating nano-transducers. "Like real ants, they produce large forces for their weight. The challenge we now face is how to control that force for nano-machinery applications," said Baumberg.

In an email exchange with New Statesman about the short-term and long-term goals in bringing this engine closer to a practical reality, Baumberg said: “It allows us for the first time, the prospect of making nano-machines and nanobots. The earliest stage applications we can see are to make pumps and valves in microfluidic systems. Microfluidic chips are really interesting for synthesising pharmaceuticals, biomedical sensing and separation, as well as many other biochemical processes.

“But all pumps and valves currently need to be made with hydraulics, so you need a pipe onto the chip for each one, limiting strongly the complexity of anything you do with them. We believe we can now make pumps and valves from the ANTs which are each controlled by a beam of light, and we can have thousands on a single chip. Beyond this, we are looking at making tiny nanomachines that can walk around, controlled by beams of light.”

The embedding of nanobots into all facets of culture is still a long way off, and researchers will need to find a way of harnessing the energy of nanoengines. However, the prospect of one day seeing the fruition of nanorobotics is worth all the patience you can get. The tiniest robot revolution has just begun.