drone mapping

Drones are fun… It is hardly news that cartography and remote sensing are moving away from heavy payload carriers such as airplanes and helicopters to smaller, unmanned systems. Commercial UAVs are now used in all kinds of sectors, not just the military.

I have been flying a DJI Spark for a few months now and I’m quite surprised at the quality of this small consumer drone when it comes to mapping. In this blog post I want to outline the drone, my workflow for processing the images and the resulting 3d maps.

With a take-off weight of approx. 300 grams, the Spark is the lightest and smallest drone of the consumer market leader dji from China. I often take it with me in a small backpack and take it out spontaneously for a flight. Flight preparation takes only about one minute: You boot the drone, then the remote control and connect your phone or tablet to the remote control.

To create a 3d scene, I fly several times around the object of my interest in a constant height and take about 50 photos. The Spark has a flight time of approx. 15 minutes per battery pack, whereby you have to subtract take-off and landing and achieve a realistic duration of 12 minutes per flight. This is sufficient to fly around even complex objects. The battery pack is easily exchangeable (I bought 3 packs so flight time is usually not a problem).

Back home I check the pictures first and delete blurred pictures manually. Then I use 3DF Zephyr, a software to reconstruct 3d models from pictures or videos. The software is quite simple, but has powerful features. Up to 50 images can be loaded and processed in the free version. In contrast to software, which I used in the past, you don’t have to manually set fixed identification points between images any more. The complete reconstruction is fully automatic. At the end, different products such as point clouds, meshes or textured models can be exported. A nice feature is the direct integration into Sketchfab, a service for hosting 3D models on the web.

Sketchfab offers the possibility to manually adjust lighting, color and contrast to create a video game like style for the models that I believe has a nice effect while still providing enough detail about the object.

Of course, not only 3d models are valuable results of a drone flight. Often it is more interesting to create orthophotos from the individual images. I intend to have a look at the Open Drone Mapping Project in the coming months and will report on the results.

Until then: Happy flying!

its / db mobility hackathon 2017

Our team consisting of four civity analyst took part in the DB / ITS Hackathon in Hamburg on Friday / Saturday and won the first place quite unexpectedly…

On a hackathon, a programming project / data analysis has to be developed from the idea to the prototype within 24 hours. We opted for the relatively current topic of “storm susceptibility of the rail infrastructure due to falling trees”. We used a combination of different methods and open datasets to create static and interactive maps to identify hot-spots within the German rail system in general and the Hamburg system in particular.

Storm risk analysis of the German railway system. Red and yellow lines indicate a heightened risk for trees falling onto the railway tracks.

You can find the finished project here: http://46.101.153.57/ 

Pictures and further information can be found at: https://dbmindbox.com/db-opendata-hackathons/hackathons/hamburg/

Let’s start blogging again

It’s been quiet here on mappable for much too long (891 days to be precise, this a data nerd blog after all). A lot of things changed since we first got started with mappable in 2013/14. Back then Mapbox and CartoDB were still brand new startups, we had all just stopped talking about google fusion tables and mapping mashups (remember that?) and putting election results on an interactive map was top notch data journalism.
A lot things changed since then. Like so many others in the field of (geospatial) data analysis and visualization, what started as a fun project in our free time grew into ordinary full-time jobs. We’re now both members of Spiegel Online’s data journalism team and Achim is leading a small data team at Hamburg/Berlin based consultancy Civity.
Unfortunately, spending all day crunching data also ment we’ve stopped write about our work here in our free time. Like a lot of other blogs, mappable.info got silent exactly the moment we’ve made most progress and probably had the most interesting work to share. Being career changers our selves and having learned so much from the open spirit in the geospatial, open source, open data & dataviz communities this is something we want to change again. We probably won’t start bigger projects here but we’ll do our best to write blog posts here every once in a while to keep you updated what we’re working on these days. Stay tuned, there’s more to come soon.

Patrick & Achim

BTW: We’ve switched our web hosts. The blog now runs on wordpress (not too happy about the move though, but we just don’t want to waste time with that any more). We’ve done our best to keep projects up an running, but there might still be some small issues and dead links.

Spinning globe with glowing city markers in D3

D3 (Data-Driven Documents) is not only one of the currently leading dataviz tool, it also has huge capabilities when it comes to mapping. Here at mappable we’ve so used it for building Travel Score and the Berlin A-Z maps. D3 has a quite steep learning curve, but since I’ve successfully worked with it while only having very basic JavaScript knowledge at that time, it’s definitely doable and in my eyes well worth the effort.

 

Not having used D3 in quite a while and a little worried that my skills would turn rusty soon, I was pleased to find this tutorial by Curran Kelleher in my twitter feed the other day. It’s a great introduction into D3 and was a welcome opportunity for me to kick start working with D3 again. Close to the end of his tutorial (here), Curran shows how to draw a world map in D3 simply by plotting graduated circle markers according to their lat/long coordinates using Geonames data. I instantly liked the idea and though it would be a good exercise to give the visualization a few additional touches, making use of D3’s extensive mapping functionalities.

First of all, I decided to use D3’s build in orthographic projection. With just a few twists and configurations I had a beautiful globe instead of a flat map. Next, I wanted to get that globe spinning. Why? Basically, because spinning globes are totally fascinating and look awesome! Again, this can be done by adding just a few lines of code.  There’s a great example by Mike Bostock showing the basic principle, which I adopted. Finally, to add some simple interactivity, I’ve integrated some HTML5 sliders and a color picker with the corresponding event listeners.

All together this gives us a spinning globe, showing all cities in the GeoNames databast with a population of more than 100.000 and handlers to manipulate rotation speed & direction, marker opacity, size & color. The result might challenge the rendering capabilities / the processing power of your computer, since it constantly re-calculates and re-projects the positions of over 4.000 markers on the globe. Looking at my machine (a 2013 MacBook Air) that amount is probably at the limit of what you should do with SVG transformations. If you want to build something bigger, working with either canvas or WebGL might be a good idea (see this OpenViz conference talk by Dominikus Bauer for a proper explanation).

Anyway, I’m quite happy with what I’ve build and wanted to share it, since I’ve learned so much just by looking at all the awesome examples out there.

The globe visualization and the source code can be found here or by clicking the image. I decided not to embed it into the blog post, but to use a animated GIF instead, since it could lag or crash browsers on weaker machines. Please also note that the visualization is only optimized for Chrome browsers. I didn’t invest any time in cross-browser compatibility.

Have fun playing with it and happy mapping with D3!

Mapping nightlife

After collecting business listings of relevant categories (nightlife, cinemas, theaters, and event locations) from a popular online directory, the requirements for mapping them were twofold:

  1. The visualizations should allow to identify hot spots of nightlife activity in each city on different map scales.
  2. They should be suitable for comparing the distribution patterns between the different cities.

We chose to tackle this challenge with static dot density maps (sometimes also called dot distribution maps), produced in QGIS. If a data set is large enough, this approach offers a much higher precission than the ordinary choropleth map or any kind of raster or hexbin map that aggregates the data. For the map styling we used cartoDB’s new Dark Matter basemap (finally a project with a good reason to chose the ever so popular dark map style) and plotted all locations on top of it. We set the circle size to 1250 meters and used a very low opacity level as well as the feature blending mode ‘screen’ to highlight those places where a lot of nightlife locations are overlapping.

You can explore the results by either browsing through the gallery on top of this blog post or, if you want to take a closer look at Berlin, Hamburg and Munich check out the high-res, zoomable maps below. As an urban planer it’s fun to see that the nightlife patterns in most cases match to the general city structure (monocentric vs. polycentric) pretty well.

Berlin

Hamburg

Munich