40 million frames per second!

40 million frames per second!

Robert Cotton / Wed, 04/04/2012 - 16:35


OK Just a factoid from our genius physicist colleagues at CERN.

Recently, I was showing students some super-slomo video taken with a Phantom HD Camera. The Phantom can shoot at 1000 frames/sec and indeed up to 2000 frames/sec on some models. Back in the days of celluloid, slow-motion (slomo) meant just 72 frames/sec. How times have changed...

But it is more than just a quantitive change. The extension of our perceptual apparatus into the world of animated suspension (for example at around 1000 frames/sec) has a profoundly different effect on our senses, and on our cognitive apparatus. The effect is an inversion of the dilated time that is often the apparent result of taking psychoactive or hallucinogenic drugs - with these drugs, time can appear to slow down, leaving us lost in wonder as we watch some simple natural process. With Phantom HD, we create a 1000 frames/sec slomo - the video, not our perception, slows down. The tiniest motion of an expectant dog waiting for a treat is made visible, made remarkable, by frame-capture at these speeds.


We see nuance upon nuance, fractional temporal detail - detail we can never see in the whirligig of realtime. Super Slomo does for cinematography what Harold Edgerton and his strobes did for stills - it captures images at the millisecond level. And the Phantom HD captures each frame at HD resolution - 2 megapixels per frame, giving us an insight into aspects of the world previously only accessible to specialist scientists or technical imaging experts.

And of course its not just the Phantom Gold HD camera that can do this - the Photron company makes several cameras of this order of performance (even one shooting 75,000 frames/sec at low-res), and IDT have their Redlake Y5 at 1500 1920x1080 frames/sec.

Then I heard the factoid that I mentioned - in an interview with the Higgs Boson-hunters at CERN, in which they described the Atlas Detector - the huge ‘camera’ that captures particle collisions in the Large Hadron Collider. This massive imaging machine can take 40 million frames/sec! And not only that, but each frame is 100 megapixels in quality! And in 3d!



When you think of the impact of Etienne Jules Marey and Edweard Muybridge in the 1880s and of Harold Edgerton’s stroboscopic high-speed photographs in the mid 20th century, it is interesting to speculate on the impact of these new 21st Century camera and imaging technologies over the next decades.

10 years ago, when the Wachowsky Brothers and SFX-director John Maeda were creating and compiling the technologies for what came to be called virtual cinematography in The Matrix, there were forecasts of realtime photogrammetry - making cameras that could shoot HD video, and also capture stereo 3d data from the real-world - both by triangulation, and by realtime automatic edge-finding and photo-grammetic interpolation of third-dimensional data. The idea that we should have cameras that recorded, processed and stored 3d model data alongside cinematographic images - that we would have something like Startrek’s Tricorder - is now very nearly a reality.

Dimensional Imaging’s 3d Content Creation Software indicates how this might be achieved from an HD stereo-lens camera: http://www.di3d.com/products/

And there’s emerging an ever-more-accessible alternative to photo-grammetry, namely the recent development of point-cloud laser scanners - now with ranges up to 120 metres. These devices emit a scatter-gun of laser-pulses, each building towards a 3-dimensional plot of all the objects in sight. This stream of data can be processed into a three-d map or model of the domain (the building, city-square, valley or other subject). Devices like the Faro point-cloud Laser Scanner already come with the CAD software for compiling and compositing scans from different perspectives (both interior and exterior) into coherent 3d CG models, that can then be coloured and textured or clad with photographic surface imagery. (see http://www.faro.com/)

Projection Mapping is another facet of this ‘virtual cinematography’ sector. This integration of 3d model data with real projection surfaces (building facades etc) has had a spectacular impact over the last decade, creating augmented realities visible to the naked eye. Other artificially created augmented realities include the delightful FLART toolkit and the Madmapper software - the use of graphic reference markers to help create 3d CGI superimpositions on real-world scenes..

see http://www.madmapper.com/gallery/

At current rates of ‘professional/scientific imaging-to-prosumer-product’ (<10 years), we might expect some interesting and affordable consumer-level cameras over the next few years.

http://www.photron.com/ - 7500 megapixel frames/sec


IDT Redlake Y5 1500 frames/sec at 1920x1080


Of course, these are developments in just one aspect of the ever-evolving digital domain. In the last decade, for example the ‘new’ has encompassed developments in bandwidth, WIFI, online encyclopedia (Wikipedia), Social Networks, MPEG-7, low-cost GPS, PS2, Tangible Media (including multi-touch), iPod, 3d Worlds (Second Life), digital pens, video networking (Skype), BitTorrent, Mars Odyssey remote-scans, iTunes, AdSense, HD video, Blogging, Augmented Reality, Lego Mindstorms, Projection Mapping, photo-sharing (Flickr), video-sharing (Youtube), DVB - Digital broadcasting, Pure Data, World of Warcraft, Ruby on Rails, Web 2.0 and Ajax, Web 3.0 (Semantic Web), Low cost computers (Arduino, One Laptop per Child, Raspberry Pi, etc) Nintendo Wii, eBooks, Modul8, Mobile TV, Google Earth, Cloud computing, Twitter, Photosynth, iPhone, Massive’s Prime crowd-animation software, Google Maps, Flip camcorders, Empathic Robotics, eBook Readers, Android, Red Digital Cameras, Tag clouds, realtime expression capture, Microvision pico projectors, BBC iPlayer, Phoenix Mars digital imaging, Crime Maps, OpenCalais, IPhone Apps Store, 3d TV, Surface Computing, Spotify, Adobe AIR, Prezi, HTML 5.0, Phantom Gold cameras, Google Chrome, iPad, Adobe Edge, WebGL, GoPro cameras, iTunesU, (to mention but a few!). Plus the Naughties were the decade of low-cost sensors, and bio-sensors, wearable computing, Big LCD and Plasma Screens, and professional social networks, and CMS like Moodle and Drupal...and open-source authoring and software development kits like Processing, Scratch, Kid’s Ruby, Corona, Pure Data, etc..

What’s the relevance of all this to ADRIART?

The pace of techno-social innovation is ever faster as processing power and digital memory obeys Moore’s Law...and The Times They Are a Changing! Designing courses for 21st century students - youngsters who already inhabit the digital world - means we need philosophies, strategies, guidelines, and courses that are fit for purpose in this new century. We must take the best from the revolutionary seminal courses, R&D centres, and design institutions of the 20th Century - Bauhaus, Black Mountain, Xerox PARC, Interval Research, MIT Media Lab, Ealing Ground Course, IVREA, the Planetary Collegium, CAL Arts, ZKM, ZHDK, and so on...And add to this best practice and experiential knowledge from all ADRIART participating institutions.

In a recent collection of essays by leading art-educators ‘Educating Artists for the Future - Learning at the Intersections of Art, Science, Technology and Culture’ (ed Mel Alexenberg, Intellect 2008), the editor notes a proposal by Daniel Pink that we are ‘witnessing a paradigm shift beyond the digital culture of the Information Age to a Conceptual Age in which people in all walks of life will succeed when they behave like artists who integrate left-brain with right-brain thinking” (p12). Other contributors suggest that although the impact of digital technology is significant, it forms just part of something much more momentous that is intertwined with the aesthetic, ethical, cultural and socio-economic. “Scientific research and technological development are radically transforming basic philosophical ideas about the nature of the physical world, time and space, the nature of life and intelligence, and the limits of our abilities to transform the world and humanity” (p13).

These and other papers by the likes of Roy Ascott, Eduardo Kac and David Gelernter argue for consideration of this general ‘post-digital’ approach, where the ‘art education’ developments are integrated into what Ascott calls the ‘moistmedia’ interplay of digital, biochemical, neurological, physical and other biological systems with the digital and cultural domains...

All these opportunities to consider, as well as getting the balance between our rich local cultures and the global developments surrounding us(!)

Perhaps we might extract from this rather list-full blog some criteria that we might apply as the ADRIART prospectus comes together over the next two years?