osm_map_processing/README.md

Overview

This project contains useful info/tools for processing OSM maps into the mapwriter file format (.map). It was developed to help generating offline map files for Locus on Android.

Dependencies / Prereqs

To run this program you'll need the following

• A Linux environment
• Python 3.x installed
• Java 1.8 or higher installed (OpenJDK works)
• wget installed
• bunzip2 installed (usually part of bzip2 package)

Installation

Head over to releases tab and download the latest. Extract the file and you're good to go. Everything is self-contained minus the above dependencies.

Running

To run the program, cd to the directory where you extracted the release and run './process_maps.py' with at least one '--map-list /path/to/list.txt parameter'. See the lists directory for examples on how to format the map list files.

Program Usage

usage: process_maps.py [-h] [--map-list MAP_LIST] [--no-sleep] [--use-ram]
                       [--max-heap-space MAX_HEAP_SPACE]
                       [--output-map-name OUTPUT_MAP_NAME]
                       [--cached-maps-dir CACHED_MAPS_DIR]

optional arguments:
  -h, --help            show this help message and exit
  --map-list MAP_LIST   a text file with one map URL per line, can be
                        specified more than once
  --no-sleep            don't sleep between downloads -- WARNING you can
                        easily run into throttling on mirrors if you use this
                        option
  --use-ram             use RAM for mapsforge processing -- WARNING mapsforge
                        uses 10x the map size in RAM for processing (ie. 100Mb
                        map = 1Gb RAM usage), you want a LOT of RAM for this
                        option
  --max-heap-space MAX_HEAP_SPACE
                        set the max heap space for the JVM, use standard -Xmx
                        values, default (4g) should be fine if not using
                        --use-ram argument
  --output-map-name OUTPUT_MAP_NAME
                        set the output .map and .poi file names
  --cached-maps-dir CACHED_MAPS_DIR
                        Use cached maps in the specified directory instead of
                        downloads using map lists

Examples

./process_maps.py --max-heap-space 24g \
    --output-map-name central_america \
    --cached-maps-dir ~/osmmapdata/cache/central_america/20180225-1429

./process_maps.py --max-heap-space 24g \
    --output-map-name midwest \
    --cached-maps-dir ~/osmmapdata/cache/midwest/20180225-1429

./process_maps.py --max-heap-space 24g \
    --output-map-name south_america \
    --cached-maps-dir ~/osmmapdata/cache/south_america/20180225-1429

Caching maps for re-use / re-processing

Included in the repo there is a download_maps.py script that is meant to help with downloading maps on a schedule or for caching. It uses the same logic as the main script but for download only.

Program usage

usage: download_maps.py [-h] [--map-list MAP_LIST] [--no-sleep]
                        [--output-map-name OUTPUT_MAP_NAME]
                        [--cached-maps-dir CACHED_MAPS_DIR]

optional arguments:
  -h, --help            show this help message and exit
  --map-list MAP_LIST   a text file with one map URL per line, can be
                        specified more than once
  --no-sleep            don't sleep between downloads -- WARNING you can
                        easily run into throttling on mirrors if you use this
                        option
  --output-map-name OUTPUT_MAP_NAME
                        set the name of the map directory before Ymd-HM
  --cached-maps-dir CACHED_MAPS_DIR
                        The root directory where maps should be cached

Example Usage

./download_maps.py --cached-maps-dir ./cache \
    --output-map-name central_america
    --map-list lists/pbf/central_america.txt

./download_maps.py --cached-maps-dir ./cache \
    --output-map-name midwest
    --map-list lists/pbf/midwest.txt

./download_maps.py --cached-maps-dir ./cache \
    --output-map-name south_america
    --map-list lists/pbf/south_america.txt

Licencing

All code is licensed Apache 2.0 and all non-code is licensed Creative Commons CC-BY-SA-3.0