I write in the evenings and in the early mornings.  During the day, I’m a Chief Information Officer.  In my case, that means I’m responsible for the data and information systems for the City of Kirkland, Washington, in the USA.  The topic of the year for people in my business has moved from “cloud” to “big data.”  For those not immersed in technology on a daily basis, that means that a lot of data is now available on the Internet.  Data is also collected in large private datasets.  This includes my Amazon shopping habits, but it also includes a lot of information about the Amazon rainforest.  This data is in what is called the cloud, which really means that a lot of it is accessible.  “Big Data” is all about making that data dance about and yield up its secrets.

This is necessary if we are going to manage our natural resources.

For example, just today, the IMF announced that it thinks energy prices are too low not only in the United States but in many other countries as well.  This conclusion was based on analysis of huge datasets about the cost of gasoline around the world, the cost of extraction and delivery, and the amount and types of tax subsidies.

In January of this year, NASA released an article that warns us about the rate of forest degradation in the Amazon because of droughts caused by climate change. The article is complete with illustrative maps.  Linked from the same page are related articles on climate change predictions and rates of de-forestation (which are getting much better).

About a month ago, I attended Microsoft Research’s Techfest “Day 0” where they let customers and press in to look at some of their work.  Imagine a grown-up version of a high-school science fair.  I watched a young man who was excited about an application he had developed.  The application mapped threats (such as poaching or loss of habitat) on top habitat locations and also identified species on the “red list” of endangered species.  It accessed multiple data sets from many locations to create the single view of the world.  A few years ago, this information would have been hard to locate, the necessary data would have been invisible to most people (hidden behind corporate and government firewalls) and it would have been almost impossible to get big enough data pipes to access the data from different places across the world.

Yesterday, I was back on the Microsoft campus and I saw Excel being used to process huge datasets without almost no secret arcane computer-speak required.

Accessible big data tools means we are on our way to being able to learn some of the many things we need to know to manage ecosystems:

• We’ll understand what happens at the edges of reserves and be better able to protect them so that natural processes inside of reserves can remain as undisturbed as possible.
• We’ll be able to show the affects of various farming experiments.  For example, we might illustrate how GMO crops actually affect other crops (good and bad), and see the affect of GMO crops on public health data and economic indicators.
• We’ll be able to track species movement due to climate change and predict possible extinctions better than we can today (which will let us make choices about whether or not to intervene, and how to intervene if we choose to).
• We’ll be able to actually model the various laws that govern private property use.  This is a first step to modifying rights to support worldwide health (I never said this would be easy!).
• We’ll be able to combine datasets in new ways, many of which we haven’t even thought about yet.  Perhaps it would be interesting to look at disease patterns compared to insect populations inside of metropolitan areas, or to compare air quality to disease vectors in mice all over the world.
• We might map algal blooms and manatee deaths in Florida against changes in currents and surface water temperatures across the world, and run that across historical data to verify assumptions and against climate models about the future to predict mass mortality in manatees.

Big data in this context is a tool for understanding the natural world. We’ll be able to grasp cause and effect more closely, and to use actual information to help us drive hard political decisions.  Yes, I know it’s not as simple as I’m making it sound.  Verifiable data quality is going to matter a lot, as is ways to share and explain the lessons we learn.  But nevertheless, I am amazed at the possibilities.  I grew up before cell phones and in the first technology job I held, we ran a whole city on less information processing capability than I have in my watch these days.

Big data is going to drive change as fast as the Internet did, and faster than other change drivers like 3D printing and nanotechnology.  Costs are plummeting.  Speed is increasing exponentially.  Humans are curious, and this is going to be a fabulous tool in the fight to maintain our planet.

Links:

Linked Brazilian Amazon Rainforest:  http://linkedscience.org/data/linked-brazilian-amazon-rainforest/

Amazon showing signs of degradation due to climate change, Nasa warns, Jonathon Watts, Guardian.co.uk, January 18, 2013:   http://www.guardian.co.uk/environment/2013/jan/18/amazon-rainforest-climate-change-nasa

National Center for Ecological Analysis and Synthesis list of Ecological and Spatial Data Sources: http://www.nceas.ucsb.edu/scicomp/data

The IUCN Redlist of Threatened Species: http://www.iucnredlist.org

IMF:  Gas prices don’t reflect true costs, NPR, March 28th: http://www.npr.org/2013/03/28/175550949/imf-gas-prices-dont-reflect-true-costs