The Null Device
Posts matching tags 'research'
2008/11/30
This is pretty cool: a researcher at Adobe has developed software for manipulating moving objects in video in real time. The software works by tracking points and assigning them to objects, and as the demo shows, allows, among other things, moving objects to be automatically "graffitied" with text or annotated with automatically moving speech/thought bubbles. It also maps the degrees of freedom of a point in an object, and by clicking and dragging on it, automatically moves the video to a frame which has that point going in the desired direction, allowing video to be "scrubbed" by dragging moving objects. He even has it compositing frames of video with different objects (people, in this case) at different positions.
2008/7/30
Researchers at Columbia University have developed a new system for protecting the privacy of people who appear in photographs. Rather than blurring or erasing faces, the new software replaces facial features with others from a library:
The software randomly selects 33,000 photos of faces from picture-sharing sites like Flickr.com, then picks the most suitable faces for each person in shot. Only the eyes, nose and mouth are used, resulting in a composite image of the two people. "It matches subject pose, lighting conditions and image resolution," says Kumar. "The selected faces are aligned to common 3D coordinates, corrected for colour and lighting, and blended into the target image."
Aside from Street View, the system could be used to obscure the faces of military personnel or eyewitnesses to crime. It could also allow amateur photographers to improve group shots, by replacing frowning faces with better photos of the same people.
(via Boing Boing) ¶ 0
2007/8/9
This is pretty impressive; a new algorithm that, when presented with a photograph with a hole cut out of it, searches a database of millions of other photographs, presents the user with a menu of similar-looking images to select from, and then composites elements of the chosen image to fill the hole seamlessly, producing an image which (in most cases) looks semantically coherent. Most impressively, it is entirely data-driven, and does not require any human-generated annotations of test data:
It uses mathematical properties of the images to make the match, and sometimes ends up serendipitously picking other images from the same location (because two photographs of, say, the Taj Mahal taken on a sunny mid-afternoon are likely to share similar properties).
Of course, it is possible to use such a tool creatively, replacing unwanted parts of an image with elements from a completely different scene, as the paper (PDF here shows:
2007/4/20
Scientists at Mitsubishi have developed a camera which allows photos to be refocussed after being taken. Unlike previous experiments in this field, this system retains the full resolution of the camera's image sensor in the resulting image. It uses a special lens with a semi-transparent patterned mask (resembling a crossword puzzle) in the middle:
Using the combination of the mask and the post-processing software, MERL researchers were able to reconstruct a 4D light field from the standard 2D camera, explained MERL Visiting Scientist Amit Agrawal. Instead of bending light rays, the patterned mask attenuates the rays inside the camera. The post-processing software reconstructs the light field using an inverse computation of the Fourier transform equation, allowing the user to refocus the image.The article has a slideshow, including some sample images. For some reason, they seem a bit dull, a bit like the images taken with digital cameras from 10 years ago; I wonder whether this is a result of the image reconstruction algorithm.
2005/1/29
Science has found that monkeys are willing to pay for monkey porn; i.e., male rhesus macaques in possession of stocks of juice will trade juice for views of female monkeys' bottoms. This emerged from a study on how much monkeys would pay to see (G-rated) pictures of monkeys of various ranks (the answer: less than for porn for high-rating monkeys, though you'd have to actually pay them to look at low-ranking loser monkeys).
Now, an interesting question would be, would it be possible to put monkeys to work (useful or experimental, as long as it's something they wouldn't normally do without reward) and pay them in a currency that's useless in and of itself but useful only for buying things (juice or monkey pr0n).
2004/4/23
An interesting item cribbed from Jim: WordsEye is an experimental system that takes sentences in English (such as "The marmalade sky. The zebra skin ground is red. The sun is large. The hippo is next to the horse.") and renders them as images (such as this one). There's a wealth of example images here. It can represent abstract ideas (such as "John believes the cat is not blue", and automatically infer how to symbolise that something happened at a certain time, and map adjectives to textures. Of course, sometimes it gets amusing results, such as its rendering of "John looks at the cat on the skateboard", or, indeed, "the devil is in the details".
2004/3/18
Real life gets a little closer to hard science fiction: NASA scientists have developed a system that responds to subvocalised commands; i.e., sentences you "speak" in your head. It does this by analysing nerve commands to the throat using sensors. It can only recognise a small fixed number of words, so it's of no use for silently talking on the phone to someone, at least not until they make it drive a speech synthesiser. (via bOING bOING)