Browse Source

Add way to handle custom polys, add notes on how to re-create openandromaps styles, etc, start basics of elevation handling

master 20180306.2
Mike C 3 years ago
parent
commit
698da0e714
  1. 19
      README.md
  2. 3923
      custom_polys/costa_rica.poly
  3. 7767
      custom_polys/nicaragua.poly
  4. 3917
      custom_polys/panama.poly
  5. 17
      custom_polys_and_extracts.md
  6. 23
      elevation.md
  7. 3
      lists/custom.txt
  8. 1
      openandromaps/.gitignore
  9. 8
      openandromaps/README.md
  10. 1205
      openandromaps/tag-mapping.min.xml
  11. 1205
      openandromaps/tag-mapping.urban.xml
  12. 1500
      openandromaps/tag-transform.xml
  13. 8
      process_maps.py
  14. 26
      pymapsforge/process.py

19
README.md

@ -27,7 +27,8 @@ To run the program, cd to the directory where you extracted the release and run
usage: process_maps.py [-h] --map-list MAP_LIST [--use-ram]
[--max-heap-space MAX_HEAP_SPACE]
[--output-map-name OUTPUT_MAP_NAME] --maps-dir MAPS_DIR
[--no-map-download]
[--no-map-download] [--tag-mapping TAG_MAPPING]
[--tag-transform TAG_TRANSFORM]
optional arguments:
-h, --help show this help message and exit
@ -45,16 +46,24 @@ optional arguments:
set the output .map and .poi file names
--maps-dir MAPS_DIR Where downloaded maps will be stored/read from
--no-map-download Do NOT download maps, re-use maps from maps-dir
--tag-mapping TAG_MAPPING
Specify a custom tag mapping xml file for use with
mapsforge processing
--tag-transform TAG_TRANSFORM
Specify a tag transform file for use PRIOR to
mapsforge processing (this is DIFFERENT than the
mapsforge mapping xml file)
```
### Examples
```
./process_maps.py --use-ram --max-heap-space 16g --maps-dir cache --no-map-download --output-map-name pennsylvania --map-list lists/pennsylvania.txt
./process_maps.py --use-ram --max-heap-space 12g --maps-dir cache --no-map-download --output-map-name idaho --map-list lists/idaho.txt
./process_maps.py --use-ram --max-heap-space 24g --maps-dir cache --no-map-download --output-map-name australia --map-list lists/australia.txt
./process_maps.py --tag-transform ./openandromaps/tag-transform.xml --tag-mapping ./openandromaps/tag-mapping.min.xml --use-ram --max-heap-space 12g --maps-dir cache --no-map-download --output-map-name custom --map-list lists/custom.txt
./process_maps.py --tag-transform ./openandromaps/tag-transform.xml --tag-mapping ./openandromaps/tag-mapping.min.xml --use-ram --max-heap-space 16g --maps-dir cache --no-map-download --output-map-name pennsylvania --map-list lists/pennsylvania.txt
./process_maps.py --tag-transform ./openandromaps/tag-transform.xml --tag-mapping ./openandromaps/tag-mapping.min.xml --use-ram --max-heap-space 12g --maps-dir cache --no-map-download --output-map-name idaho --map-list lists/idaho.txt
./process_maps.py --tag-transform ./openandromaps/tag-transform.xml --tag-mapping ./openandromaps/tag-mapping.min.xml --use-ram --max-heap-space 24g --maps-dir cache --no-map-download --output-map-name australia --map-list lists/australia.txt
./process_maps.py --use-ram --max-heap-space 24g --maps-dir cache --no-map-download --output-map-name canada_ontario --map-list lists/ontario.txt
./process_maps.py --max-heap-space 8g --maps-dir cache --no-map-download --output-map-name midwest-2 --map-list lists/midwest.txt --map-list lists/ontario.txt
./process_maps.py --tag-transform ./openandromaps/tag-transform.xml --tag-mapping ./openandromaps/tag-mapping.min.xml --max-heap-space 8g --maps-dir cache --no-map-download --output-map-name midwest-2 --map-list lists/midwest.txt --map-list lists/ontario.txt
./process_maps.py --max-heap-space 8g --maps-dir cache --no-map-download --output-map-name usa_northeast --map-list lists/usa_northeast.txt
./process_maps.py --max-heap-space 8g --maps-dir cache --no-map-download --output-map-name usa_pacific --map-list lists/usa_pacific.txt
./process_maps.py --max-heap-space 8g --maps-dir cache --no-map-download --output-map-name usa_south --map-list lists/usa_south.txt

3923
custom_polys/costa_rica.poly
File diff suppressed because it is too large
View File

7767
custom_polys/nicaragua.poly
File diff suppressed because it is too large
View File

3917
custom_polys/panama.poly
File diff suppressed because it is too large
View File

17
custom_polys_and_extracts.md

@ -0,0 +1,17 @@
# Creating custom polys (some rough notes)
Head to https://www.openstreetmap.org/ and find the relation id of the boundry you want as a poly
Grab an appropriate pbf/osm.bz2 from http://download.geofabrik.de/ that INCLUDES the region you want to extract if not available
Head to http://polygons.openstreetmap.fr/index.py and enter relation id
Extract the region using osmosis
https://wiki.openstreetmap.org/wiki/Osmosis/Examples
http://blog.geofabrik.de/?p=75
../bin/osmosis/bin/osmosis --read-pbf central-america-latest.osm.pbf \
--tee 3 \
--bp file=costa_rica.poly --write-pbf costa-rica-latest.osm.pbf \
--bp file=nicaragua.poly --write-pbf nicaragua-latest.osm.pbf \
--bp file=panama.poly --write-pbf panama-latest.osm.pbf

23
elevation.md

@ -0,0 +1,23 @@
# Elevation Data
To add elevation data and contour lines to your maps, you'll need the below information.
# Credits
The information here and integration(s) in the main ```process_maps.py``` script were adapted from:
* http://forum.locusmap.eu/index.php?topic=1839.0
* https://www.openandromaps.org
# Prep
* You'll need a theme from : https://www.openandromaps.org/en/legend/elevate-mountain-hike-theme
* You'll need tag mappings : https://www.openandromaps.org/en/map-basics-2/tag-mapping
* You'll probably want tag transformations : https://www.openandromaps.org/en/map-basics-2/tagtransform
# Processing
1. Download various elevation data needed using phyghtmap (http://katze.tfiu.de/projects/phyghtmap/phyghtmap.1.html)
1. Download maps using ```download_maps.py```
1. Create a custom map list (see ```custom.list``` for an example) for use
1. Run map processing with ```--no-map-download```, ```--tag-mapping``` and ```--tag-transform```

3
lists/custom.txt

@ -0,0 +1,3 @@
custom/costa-rica
custom/nicaragua
cusom/panama

1
openandromaps/.gitignore

@ -0,0 +1 @@
Elevate*

8
openandromaps/README.md

@ -0,0 +1,8 @@
# Additional Processing
If you're interested in additional processing applied by www.openandromaps.org you'll want to look at the following links.
The additional tag processing in particular has been helpful for map usage in Locus
* https://www.openandromaps.org/en/map-basics-2/tagtransform
* https://www.openandromaps.org/en/map-basics-2/tag-mapping

1205
openandromaps/tag-mapping.min.xml
File diff suppressed because it is too large
View File

1205
openandromaps/tag-mapping.urban.xml
File diff suppressed because it is too large
View File

1500
openandromaps/tag-transform.xml
File diff suppressed because it is too large
View File

8
process_maps.py

@ -31,6 +31,10 @@ if __name__ == '__main__':
help='Where downloaded maps will be stored/read from')
parser.add_argument('--no-map-download', action='store_true',
help='Do NOT download maps, re-use maps from maps-dir')
parser.add_argument('--tag-mapping', action='store',
help='Specify a custom tag mapping xml file for use with mapsforge processing')
parser.add_argument('--tag-transform', action='store',
help='Specify a tag transform file for use PRIOR to mapsforge processing (this is DIFFERENT than the mapsforge mapping xml file)')
args = parser.parse_args()
current_timestamp = datetime.datetime.now().strftime('%Y%m%d-%H%M')
@ -44,6 +48,8 @@ if __name__ == '__main__':
process.prep_land_sea_polys(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name)
process.process_land_sea_polys(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name, args.max_heap_space, args.use_ram)
process.osmosis_main(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name, args.max_heap_space, args.use_ram)
if args.tag_transform is not None:
process.process_tag_transform(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name, args.max_heap_space, args.use_ram, args.tag_transform)
process.osmosis_main(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name, args.max_heap_space, args.use_ram, args.tag_mapping)
process.osmosis_pois(base_path, current_timestamp, args.maps_dir, args.map_list, args.output_map_name, args.max_heap_space)

26
pymapsforge/process.py

@ -93,7 +93,29 @@ def process_land_sea_polys(base_path, current_timestamp, maps_dir, map_lists, ou
cmd = subprocess.Popen(osmosis_cmd, env=env)
cmd.wait()
def osmosis_main(base_path, current_timestamp, maps_dir, map_lists, output_map_name, max_heap_space, use_ram):
def process_tag_transform(base_path, current_timestamp, maps_dir, map_lists, output_map_name, max_heap_space, use_ram, tag_transform):
print('Transforming tags prior to final map writing')
output_dir = current_timestamp + '-' + output_map_name
cached_maps_dir = os.path.abspath(maps_dir)
poly_map = os.path.join(base_path, 'out', output_dir, output_map_name) + '_merged_polys.pbf'
# Setup various runtime aspects (going to do multiple osmosis runs (maps AND POIs)
env = os.environ.copy()
env['JAVACMD_OPTIONS'] = '-Xmx' + max_heap_space + ' -server -Djava.io.tmpdir=' + os.path.join(base_path, 'tmp') # Setup java temp dir to something a bit more sane (tmpfs /tmp for the loss)
osmosis_cmd = [os.path.join(base_path, 'bin', 'osmosis', 'bin', 'osmosis')]
osmosis_cmd.extend(['--rb', 'file=' + poly_map])
osmosis_cmd.extend(['--tag-transform', 'file=' + os.path.abspath(tag_transform)])
osmosis_cmd.extend(['--log-progress'])
osmosis_cmd.extend(['--wb', 'file=' + poly_map + 'tt.pbf'])
osmosis_cmd.extend(['omitmetadata=true'])
cmd = subprocess.Popen(osmosis_cmd, env=env)
cmd.wait()
shutil.move(poly_map, poly_map + '.orig')
shutil.move(poly_map + 'tt.pbf', poly_map)
def osmosis_main(base_path, current_timestamp, maps_dir, map_lists, output_map_name, max_heap_space, use_ram, tag_mapping):
output_dir = current_timestamp + '-' + output_map_name
bounds = poly.calculate_total_bounds(maps_dir, map_lists)
@ -115,6 +137,8 @@ def osmosis_main(base_path, current_timestamp, maps_dir, map_lists, output_map_n
osmosis_cmd.extend(['type=hd'])
osmosis_cmd.extend(['bbox=' + str(bounds['bottom']) + ',' + str(bounds['left']) + ',' + str(bounds['top']) + ',' + str(bounds['right'])])
osmosis_cmd.extend(['threads=1', 'map-start-zoom=8'])
if tag_mapping is not None:
osmosis_cmd.extend(['tag-conf-file=' + os.path.abspath(tag_mapping)])
cmd = subprocess.Popen(osmosis_cmd, env=env)
cmd.wait()

Loading…
Cancel
Save