Rough sketches as I think about how to make the particles interact with the video mesh scene.
Technical Progress
Experiments
Using a hacked web-cam I experimented using IR camera.
There is some quality issues with the hacked camera. I think a higher resolution camera would have better results. See the left image vs my hacked web camera on the right.
Research
References:
- http://www.creativeapplications.net/tutorials/guide-to-camera-types-for-interactive-installations/
- http://www.flong.com/blog/2010/a-brief-note-on-infrared-filters-87-vs-87c/
- http://openframeworks.cc/ofBook/chapters/image_processing_computer_vision.html
Equipment:
New Prototypes:
Pipeline Updates:
- Researched IR and thermal cameras but still not sure which camera is best suited given time, money and tech specs. Miguel is meeting with the MIT PI from Camera Culture tomorrow to get a better sense of the best camera for our needs.
- I need to do some stitching together of code to get all the video distortions complete
- I need to use GPU particles systems rather than CPU systems to get the "edges" of the video mesh to "speak" to the particles
Inspiration
http://internet-atlas.net/
Flow Fields in Three.js
“What is a flow field? Think of your Processing window as a grid. In each cell of the grid lives an arrow pointing in some direction—you know, a vector. As a vehicle moves around the screen, it asks, ‘Hey, what arrow is beneath me? That’s my desired velocity!’”
Chapter 6.6 Flow Fields: http://natureofcode.com/book/chapter-6-autonomous-agents/
We are using Flow Fields to elucidate the environment with the movements of the microbes.
I am using a paper called "Computational Fluid Dynamics of Sponge Aquiferous Systems" as a scientific reference and pulling some artistic license as to how microbes may move in an urban environment.
Mapping Microbes using Thermal Imaging



































Design Meeting - March 2016
Our task is to come up with design solutions for these three elements:
- Represent the environment
- Represent the microbes within that (protagonists)
- Perceptual apparatus - how to show in the exposition
Representing the Environment
How do microbes experience the world?
- sense of smell
- chemical interactions
- chemical compositions: asphalt, wood, soil, cement
Katamari -> Hives -> Things are adhering to it -> We are interested in the whole, things interacting together, not in the individual compnents.
Perfume the film --> Tools: how to see air?
Perceiving the chemical element of the environment --> chemical composition of the environment
How to map chemicals to pixels?
This project uses bees as sensors, as citizen scientists - so this should not be focused on the bees, but on the microbes.
In the context of architecture:
- How can you design for microbes in mind?
- Architecture at the micro scale
- What are the micro environments? Textures: wood, asphalt, etc.
- Micro textures
- Topologies of the micro scale
We want to have a dialog with the invisible - if the table could talk, what would it say about its microbes?
Flow fields
https://www.chromeexperiments.com/experiment/fluid-and-particles-in-webgl
We discussed at length about using flow fields to represent the movement of the microbes in space using this scientific paper as reference: LINK TO PAPER
Flow going around the objects - a car comes and swipes away the particles -> Microbial Flow Area
Sensory Humoculous
Remapping the world from the microbes perspective
How to volumetrically map the world?
References:
- http://bengler.no/terrafab
- http://www.wired.com/2015/04/laser-scans-london-new-way-see-world/
- http://berglondon.com/products/hat/
- http://www.specular.cc/
- http://www.depthkit.tv/
Ideas:
- Volumetrically recording an area --> like 1 street -> like Google street view if the "car" was a microbe
- Expand edges and shrink surfaces
- Textures - accentuating the textures
Main ideas that surfaced:
- Flow --> Travel (macro)
- Crevasses --> the microbe's home (micro)
Flow
We decided to go with the flow idea.
Looking at the city at large scale
Spacial information
Taxonomical Mapping
Notes on Mandelbrot:
https://www.youtube.com/watch?v=x81lv4xfyNo
http://www.fractalforums.com/3d-fractal-generation/true-3d-mandlebrot-type-fractal/
http://tilde.club/~david/m/#zoom=6.292011834319527,3.4&lookAt=-0.6,0&iterations=85&superSamples=1&escapeRadius=10.0&colorScheme=pickColorGrayscale
https://github.com/coded-aesthetics/mandelbro
http://www.coded-aesthetics.com/
https://processing.org/examples/mandelbrot.html
http://blog.hvidtfeldts.net/index.php/generative-art-links/
http://sub.blue/fractal-lab
http://davideriboli.isiaurbino.net/arte-generativa
INFOGRAPHIC INSPIRATION
Diatoms:
Microbe Inspiration
Video Manipulating - ofxBlur
I am using ofxBlur to render the image only when there is movement in the camera.
First I took a video in the street using ofxBlur:
Then I run the blur video through the three.js video tool using Edge Detection and Blur effect:
Then I ran that video again through my tool using the mesh distortion and the edge detection:
Hyperlaspe.js
Hyperlapse.js library with three.js --> mapping the video to a 3D plane geometry with 3D particles being able to fly through the plane. This is generative so you can go anywhere in the world: New York, Venice, Australia, Tokyo.
Thesis II - Update
“There is a vast negative space around us, and we think it’s empty, the same way we imagined life before germ theory. It’s not empty. It’s more alive than anything we build. This is a project in shifting what Foucault called “the liminal horizon” so that you understand the whole world differently when you look away from what we’ve shown you.”
FINAL PIECE CONCEPT & FORM:
My final piece for the thesis show and main representation of my research will be a web-based visualization tool that is meant to be an evocative art piece. I am intending for this piece to capture the imagination of the audience.
I am building a web-based experience that is a cross between a film, video-game, and data visualization, enabling the user to imagine the microbial nature of the city.
I am asking the user to reimagine what it means to be human in the vast metropolis that is New York City. Who are the “companion species” that live among us? How might the world appear from a microbes point of view?
The web site will be built out to include information about the data collection, analysis and preliminary findings based on the data-set from New York City.
CURRENT PROTOTYPE (MAR 2016):
I am using three.js to build a custom web-tool. The user will be able to select a path or journey through the city (say 5th Ave and 10th St, to 5th Ave and Washington Square Park). Then using hyperlapse.js and Google Street Images, a video of the path will be made. The video is then mapped to a 3D mesh in 3D space. Various custom filters (which are part of my visualization tool-kit) can be applied to the video to make the city appear as if the microbial world is coming to life. Finally 3D particle systems are applied to the 3D video that further seek to illustrate the microbial nature of New York City.
Three.js with Particles
Adding particles into the scene in three.js.
Three.js Video Manipulation
Ideation II
Using thermal cameras to show the human body - applying particles to a scene that is mapped by thermal radiation:
Some kind of "invisible" ink only visible when shown under UV light:
Thermochromatic paper:
Mapping shapes to microbes -- a "design taxonomy" for the microbes:
There is something aesthetically that I love about Sol Lewitt and his contemporaries. Something about the micro and the macro that has always been very appealing to me.
Sol Lewitt Wall Drawing 1180
Biological Immaterials - Ideation II
Peering into the microbial world
I visited the Cooper Union lab to look at some microbes:
Biological Immaterials - Ideation
I borrowed a small microscope to begin looking into the microbial world:
Experiments with a micropscope
PERFORMANCE
GOOGLE EARTH & MICROSCOPIC
MICROSCOPIC TEXTURES
GOOGLE STREET VIEW
KALAIDESCOPE