# Azure Run Notes

### Tested on

* Azure Virtual Machine
* E8S_V3 Standard Sized VM
  * 8 vCPUs
  * 64Gb RAM
  * 128Gb local SSD
* Azure blob storage for map data

# Post install / first login

```
wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt update
sudo apt install git tmux vim openjdk-8-jre python3 htop iotop nload
sudo apt upgrade
sudo systemctl reboot
```

# Setup Azure blob storage

### How to blob storage @ CLI
https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux

### Setup VM

```
wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install blobfuse
```

### Configure blob storage in VM

```
vim ~/blob_fuse_connection.cfg
accountName myaccount
accountKey myaccesskey==
containerName mycontainer
```

### Setup various filesystem dependencies / zones

```
mkdir /home/ubuntu/osmmapdata
sudo chown: /home/ubuntu/osmmapdata
sudo mkdir /mnt/resource/blobfusetmp
sudo chown ubuntu: /mnt/resource/blobfusetmp
```

### Mount blob storage for use

```
blobfuse /home/ubuntu/osmmapdata \
    --tmp-path=/mnt/resource/blobfusetmp \
    --config-file=/home/ubuntu/blob_connection.cfg \
    -o attr_timeout=240 \
    -o entry_timeout=240 \
    -o negative_timeout=120 \
    --file-cache-timeout-in-seconds=0 \
    -o allow_root
```

### View on Windows Desktop
https://azure.microsoft.com/en-us/features/storage-explorer/

# Run OSM Map Processing

### Prep session
```
tmux new -s maps
git clone https://github.com/mcrosson/osm_map_processing.git
cd osm_map_processing
```

### Download maps
Download maps using lists to local cache (use a blob storage endpoint, these can get very large)

```
# Example, reformat to your needs/desires
./download_maps.py --map-list lists/test.txt --cached-maps-dir ./cache --output-map-name test
```

### Midwest USA

```
./process_maps.py --use-ram --max-heap-space 48g \
    --output-map-name midwest \
    --cached-maps-dir ~/osmmapdata/cache/midwest/20180225-1429
```

### South America
```
./process_maps.py --use-ram --max-heap-space 48g \
    --output-map-name south_america \
    --cached-maps-dir ~/osmmapdata/cache/south_america/20180225-1429
```

### Central America

*NOTE: This CAN NOT fit into 64Gb RAM, you'll need to setup a 128Gb SSD/Disk at ~/osm_map_processing/tmp for this to not enounter heap space errors* 

```
./process_maps.py --max-heap-space 48g \
    --output-map-name central_america \
    --cached-maps-dir ~/osmmapdata/cache/central_america/20180225-1429
```