108 lines
2.6 KiB
Markdown
108 lines
2.6 KiB
Markdown
# Azure Run Notes
|
|
|
|
### Tested on
|
|
|
|
* Azure Virtual Machine
|
|
* E8S_V3 Standard Sized VM
|
|
* 8 vCPUs
|
|
* 64Gb RAM
|
|
* 128Gb local SSD
|
|
* Azure blob storage for map data
|
|
|
|
# Post install / first login
|
|
|
|
```
|
|
wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb
|
|
sudo dpkg -i packages-microsoft-prod.deb
|
|
sudo apt update
|
|
sudo apt install git tmux vim openjdk-8-jre python3 htop iotop nload
|
|
sudo apt upgrade
|
|
sudo systemctl reboot
|
|
```
|
|
|
|
# Setup Azure blob storage
|
|
|
|
### How to blob storage @ CLI
|
|
https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux
|
|
|
|
### Setup VM
|
|
|
|
```
|
|
wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb
|
|
sudo dpkg -i packages-microsoft-prod.deb
|
|
sudo apt-get update
|
|
sudo apt-get install blobfuse
|
|
```
|
|
|
|
### Configure blob storage in VM
|
|
|
|
```
|
|
vim ~/blob_fuse_connection.cfg
|
|
accountName myaccount
|
|
accountKey myaccesskey==
|
|
containerName mycontainer
|
|
```
|
|
|
|
### Setup various filesystem dependencies / zones
|
|
|
|
```
|
|
mkdir /home/ubuntu/osmmapdata
|
|
sudo chown: /home/ubuntu/osmmapdata
|
|
sudo mkdir /mnt/resource/blobfusetmp
|
|
sudo chown ubuntu: /mnt/resource/blobfusetmp
|
|
```
|
|
|
|
### Mount blob storage for use
|
|
|
|
```
|
|
blobfuse /home/ubuntu/osmmapdata \
|
|
--tmp-path=/mnt/resource/blobfusetmp \
|
|
--config-file=/home/ubuntu/blob_connection.cfg \
|
|
-o attr_timeout=240 \
|
|
-o entry_timeout=240 \
|
|
-o negative_timeout=120 \
|
|
--file-cache-timeout-in-seconds=0 \
|
|
-o allow_root
|
|
```
|
|
|
|
### View on Windows Desktop
|
|
https://azure.microsoft.com/en-us/features/storage-explorer/
|
|
|
|
# Run OSM Map Processing
|
|
|
|
### Prep session
|
|
```
|
|
tmux new -s maps
|
|
git clone https://github.com/mcrosson/osm_map_processing.git
|
|
cd osm_map_processing
|
|
```
|
|
|
|
### Download maps
|
|
Download maps using lists to local cache (use a blob storage endpoint, these can get very large)
|
|
|
|
```
|
|
# Example, reformat to your needs/desires
|
|
./download_maps.py --map-list lists/test.txt --map-dir ./cache
|
|
```
|
|
|
|
### Midwest USA
|
|
|
|
```
|
|
./process_maps.py --use-ram --max-heap-space 50g --maps-dir cache --no-map-download --output-map-name midwest --map-list lists/midwest.txt --map-list lists/ontario.txt
|
|
```
|
|
|
|
### South America
|
|
```
|
|
|
|
./process_maps.py --use-ram --max-heap-space 50g --maps-dir cache --no-map-download --output-map-name south_america --map-list lists/south_america.txt
|
|
```
|
|
|
|
### Central America
|
|
|
|
*NOTE: This CAN NOT fit into 64Gb RAM, you'll need to setup a 128Gb SSD/Disk at ~/osm_map_processing/tmp for this to not enounter heap space errors*
|
|
|
|
```
|
|
./process_maps.py --max-heap-space 50g --maps-dir cache --no-map-download --output-map-name central_america --map-list lists/mexico.txt --map-list lists/central_america.txt
|
|
```
|
|
|