diff --git a/azure_notes.md b/azure_notes.md new file mode 100644 index 0000000..fa1d105 --- /dev/null +++ b/azure_notes.md @@ -0,0 +1,109 @@ +# Azure Run Notes + +### Tested on + +* Azure Virtual Machine +* E8S_V3 Standard Sized VM + * 8 vCPUs + * 64Gb RAM + * 128Gb local SSD +* Azure blob storage for map data + +# Post install / first login + +``` +wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb +sudo dpkg -i packages-microsoft-prod.deb +sudo apt update +sudo apt install git tmux vim openjdk-8-jre python3 htop iotop nload +sudo apt upgrade +sudo systemctl reboot +``` + +# Setup Azure blob storage + +### How to blob storage @ CLI +https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux + +### Setup VM + +``` +wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb +sudo dpkg -i packages-microsoft-prod.deb +sudo apt-get update +sudo apt-get install blobfuse +``` + +### Configure blob storage in VM + +``` +vim ~/blob_fuse_connection.cfg +accountName myaccount +accountKey myaccesskey== +containerName mycontainer +``` + +### Setup various filesystem dependencies / zones + +``` +mkdir /home/ubuntu/osmmapdata +sudo chown: /home/ubuntu/osmmapdata +sudo mkdir /mnt/resource/blobfusetmp +sudo chown ubuntu: /mnt/resource/blobfusetmp +``` + +### Mount blob storage for use + +``` +blobfuse /home/ubuntu/osmmapdata \ + --tmp-path=/mnt/resource/blobfusetmp \ + --config-file=/home/ubuntu/blob_connection.cfg \ + -o attr_timeout=240 \ + -o entry_timeout=240 \ + -o negative_timeout=120 \ + --file-cache-timeout-in-seconds=0 \ + -o allow_root +``` + +### View on Windows Desktop +https://azure.microsoft.com/en-us/features/storage-explorer/ + +# Run OSM Map Processing + +### Prep session +``` +tmux new -s maps +git clone https://github.com/mcrosson/osm_map_processing.git +cd osm_map_processing +``` + +### Download maps +``` +# Download relevant maps using upcoming download util to ~/osmmapdata/cache/blah/date-time +``` + +### Midwest USA + +``` +./process_maps.py --use-ram --max-heap-space 48g \ + --output-map-name midwest \ + --cached-maps-dir ~/osmmapdata/cache/midwest/20180225-1429 +``` + +### South America +``` +./process_maps.py --use-ram --max-heap-space 48g \ + --output-map-name south_america \ + --cached-maps-dir ~/osmmapdata/cache/south_america/20180225-1429 +``` + +### Central America + +*NOTE: This CAN NOT fit into 64Gb RAM, you'll need to setup a 128Gb SSD/Disk at ~/osm_map_processing/tmp for this to not enounter heap space errors* + +``` +./process_maps.py --max-heap-space 48g \ + --output-map-name central_america \ + --cached-maps-dir ~/osmmapdata/cache/central_america/20180225-1429 +``` +