MapBrief™

As a buzz-worthy term “Big Data” has a lot going for it: easy to remember, vague enough that a shared clear meaning is always in doubt, and its own O’Reilly conference. Like programmers describing the merits of their software only in terms of the number of lines of code, talking about big-ness merely in terms of number of records in a database seems to be missing some larger point.

Though we in the geo world have experience with bulky data (e.g. LIDAR, maybe SCADA…), what’s looming with the sensor web, ubiquitous GPS, etc. is on a different scale altogether. One would hope that given our advantage of being schooled in the analysis and display of location our industry would have more than a leg up on our uninitiated brethren. But then haven’t we already seen the story  of companies who succeeded by understanding scale then figuring out the geo part later?

Paradoxically enough, success with Big Data may be as much a question of Art as of Science. Because the phrase “the data tells the story”–which was never true–is even more misleading in the context of Big Data due to its size and speed. A common analogy is that of sticking one’s face in front of an open fire hydrant: the expectation of the data telling its own story and you’d emerge a bit dazed, only able to conclude that you experienced some sort of odorless liquid.

                      Big Data is unwieldy and difficult to bend to one's will

Context and narrative are key no matter what data you’re dealing with, but without it Big Data in particular has little value. To use another analogy, the value of Big Data is directly correlated to an organization’s ability to mine big data for meaningful, actionable information.  The decidedly mixed record of the enterprise in doing anything interesting with their data besides storage, retrieval, and elementary reporting fueled this great take that there “There’s no such thing as big data.” That’s why Michael Driscoll sees “Big Analytics” as a necessary complement to Big Data. This is where geo can shine: there is no more immediate context than location context; throw in spatial analysis and now you’re cooking with Crisco®.

~  ~  ~  ~  ~  ~  ~  ~  ~

So Big Data requires more than adding a couple of sub-select statements to your trusty SQL queries.  Parallel processing strategies, NoSQL databases, and much faster methods of moving data from server to browser (Node.js) are some of the weaponry needed to tame the beast. Now I mentioned O’Reilly’s Strata Conference in New York in September: who doesn’t feel smarter and better-looking at an O’Reilly conference despite (or because of) its $3245 price tag?  But I have a better deal:  how about high-fiber, roll-your-sleeves-up, geospatial-centric sessions on Big Data scalability, turbo-charged spatial analysis, and moving all these bits and bytes around in the web world for one-third the cost!  Now, truth in advertising, we’re a homely bunch, but that will improve after a few craft brews at our Mile High altitude. If money is no object, how player would it be to hit up FOSS4G in Denver, then be primed to drop the knowledge amongst the black-clad, rimless-specs set at Strata New York two weeks later?

Answer: very player.

—Brian Timoney

With the FOSS4G conference in Denver five weeks away, one takeaway is already clear:  geospatial open source is drawing serious interest from a wider variety of sectors than we have seen in the past. My hunch is that in an ever-more competitive landscape with straitened budgets, there are three elements in particular that exert a broad appeal: technology, talent, and ideas.

Have a gander at the sponsor page: the presence of heavyweights ESRI, Google, and Oracle could be construed any number of ways. From scouting out emerging technology that may pose a competitive threat to looking for developer and managerial talent who know the geospatial realm, there is plenty of unique value that may not be showing up at their own large events.

But check out Newmont Mining:  the Fortune 500 company is not only a conference sponsor  but is also conducting a tutorial on rolling a cloud-based geo-collaboration system based on their own experience of using open source to coordinate their far flung international operations. Now that your eyebrow is raised, consider the presence of the NGA (National Geospatial-Intelligence Agency) who will be laying out its plan for a significantly more prominent role of geospatial open source in their operations.

Winds of change indeed.

It won't just be the usual suspects at the FOSS4G conference in Denver this September

Scanning the RSVP list to date yields more of the same: the premier federal IT contractors, a couple of large environmental firms, the premier statistical analysis software outfit (and a hot analysis/visualization start-up), a half-dozen of the biggest federal agencies, and a well known insurance company.

What accounts for this recent, broad acceptance?  While others have traced the evolution of proprietary vs. open source in the geo market, I suspect the thorniest (and potentially most profitable) challenges today are the ones that don’t lend themselves to “Next » Next » Finish” answers.  Shrink-wrapped software is very good at solving shrink-wrapped problems, and leaving “solutions” at the mercy of a vendor’s software update schedule is a competitive disadvantage.

Check out the program: there is content geared towards all levels of technical ability, including a day-long primer for newcomers that will directly address the business side of geospatial open source.

We’ll see you in Denver.

—Brian Timoney

In the aftermath of last week’s post describing how ESRI’s push into the Cloud will have a major impact on how third-party integrators and consultants will make money, there was some interesting back-channel conversation dealing with what exactly are the components of a geospatial business model and what they might look like in a more web-centric universe.

An under-appreciated aspect of the web mapping revolution has been the loss of user-intimacy with particular brands that very much holds sway in the desktop realm.  With traditional GIS being a somewhat complicated technology, getting anything done  requires a significant investment in learning a particular vendor’s desktop software package to the extent that the switching costs between ESRI, MapInfo, and Autodesk are quite high. On the web, where the ease of user experience trumps all (and Facebook, Angry Birds, et al, are a mere mouse-click away), non-professional users are largely unaware of the brands providing the experience–with the big fat exception of Google, particularly Google Earth.

On the web the goal is fast, intuitive, and accurate: the end user doesn’t care how you make it happen.

Open source geospatial software provides healthy building blocks to profitable business models in the web world

That’s why geospatial open source has shined during the web mapping revolution:  the ability to customize the end-user experience as needed using components such as GeoServer, MapServer, PostGIS, OpenLayers, etc. (often in hybrid settings alongside vendor-specific components).  A key enabler of the interoperability that fitting these pieces and parts together implies has been OGC open standards such as WMS, KML, etc.  As online maps evolve beyond being mere viewers, heavier analytic processing is happening on web servers, often via the OGC Web Processing Services (WPS) protocol.

Never heard of WPS?  This year’s FOSS4G conference in Denver will have over 20 sessions addressing the topic in some fashion or another.

And that speaks to the value of open source:  a bazaar of established proven technologies, new approaches to longstanding challenges, and edgy stuff that raise as many questions as answers. How do these different components can accomplish your clients’  business needs?  That’s where your experience and creativity comes into play.  Keeping with the food analogy, what’s on offer at this year’s FOSS4G conference in Denver are a full survey of quality ingredients and a wide variety of tasty dishes that we’re cooked up to solve specific problems.  Because open source isn’t created in a vacuum: folks go to the trouble because there’s a problem they want solved (thus minimizing those uncomfortable moments at every vendor conference where some ‘solution’ is being demo’d that bears zero relation to any actual workflow known to the audience).  To be sure, like any buffet you’ll run across items that will leave you wondering What was that? But isn’t that the creative process:  hits, misses, and stuff so out there you have to track creator down and ask “what were you thinking?”

And they’ll tell you; and give you a link to the source code.

Browse the program and ask yourself if there are enough nuggets there that warrant further investigation.  If you need a primer to get up to speed plus a brass-tacks discussion of how folks are making money with open source, check out our day-long introductory event with some of the geospatial industry’s heaviest hitters, which you can attend as a prelude to the main sessions or as a standalone event.

Either way, we’ll see you in Denver in September.

—Brian Timoney