diff --git a/README.md b/README.md index 3148e45..50440db 100644 --- a/README.md +++ b/README.md @@ -4,19 +4,26 @@ The Lollipop Cloud project is a way to self-host internet services, such as a we (The above blurb came from [our website (link)](https://lollipopcloud.solutions/), in case you're just joining us.) +## Some Links Relevant to the Project + +- [Code of Conduct (link)](code-of-conduct.md) +- [Contributing guidelines (link)](contributing/) +- [License (link)](LICENSE.md) + ## Recommended Hardware -* A [single-board computer (SBC) (link)](https://en.wikipedia.org/wiki/Single-board_computer) like the [Orange Pi (link)](http://www.orangepi.org/) or [Odroid (link)](https://www.hardkernel.com/main/main.php) with [Armbian](https://www.armbian.com/) installed. Any SBC with 512MB RAM or better, and an arm32v7 or arm64v8 chip chip should be sufficient. - * **Note about the Raspberry Pi:** The [Raspberry Pi 3 (link)](https://www.raspberrypi.org/documentation/hardware/raspberrypi/README.md) with the BCM2837 processor, at least 512MB RAM, and Raspbian may work well as a Lollipop; however, we have not tested this configuration at this time. -* A means to work with the SBC (monitor/keyboard, serial adapter, ssh via ethernet). -* A wifi adapter, USB cellular modem (3G/4G/WCDMA/LTE), [GPS module (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/gps.md), [RTC (Real Time Clock) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/rtc.md), [Bluetooth Serial Terminal (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/bluetooth_terminal.md), other hardware you need for your custom setup. See also: [Hardware Considerations (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware_notes.md). +- A [single-board computer (SBC) (link)](https://en.wikipedia.org/wiki/Single-board_computer) like the [Orange Pi (link)](http://www.orangepi.org/) or [Odroid (link)](https://www.hardkernel.com/main/main.php) with [Armbian](https://www.armbian.com/) installed. Any SBC with 512MB RAM or better, and an arm32v7 or arm64v8 chip chip should be sufficient. + - We have generated Armbian builds specifically for the Lollipop's supported boards. You can [download Armbian builds for Lollipops here (link)](https://dl.lollipopcloud.solutions/files/armbian). + - **Note about the Raspberry Pi:** The [Raspberry Pi 3 (link)](https://www.raspberrypi.org/documentation/hardware/raspberrypi/README.md) with the BCM2837 processor, at least 512MB RAM, and Raspbian may work well as a Lollipop; however, we have not tested this configuration at this time. +- A means to interact with the SBC (monitor/keyboard, serial adapter, or ssh via ethernet). +- A wifi adapter, USB cellular modem (3G/4G/WCDMA/LTE), [GPS module (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/gps.md), [RTC (Real Time Clock) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/rtc.md), [Bluetooth Serial Terminal (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware/bluetooth_terminal.md), other hardware you need for your custom setup. See also: [Hardware Considerations (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/hardware_notes.md). ## Initial Setup [Setup For Beginners (link)](for-beginners.md): If you are new to installing Linux, or would like a little extra help walking through the initial creation of your Lollipop, please take a look our documentation written especially for beginners. Then skip down to the next section ([Networking](#Networking)) when you're ready. If you're not using the [link for beginners](for-beginners.md), the following links are good first steps: -* [Download Armbian (link)](https://dl.lollipopcloud.solutions/files/armbian) -* [First Boot (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/first_boot.md) -* [Base Setup (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/base_setup.md) +- [Download Armbian (link)](https://dl.lollipopcloud.solutions/files/armbian) +- [First Boot (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/first_boot.md) +- [Base Setup (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/base_setup.md) ## Networking Setup @@ -26,28 +33,28 @@ If you're not using the [link for beginners](for-beginners.md), the following li - [FirewallD (firewall/routing) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/firewalld.md) - [Unbound (Non ISP DNS) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/unbound.md) - VPN Options - - [Private Internet Access (VPN) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/pia.md) - - [vpn.ac (VPN) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/vpn_ac.md) - - [VPN Autoconnect (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/vpn_autoconnect.md) + - [Private Internet Access (VPN) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/vpn/pia.md) + - [vpn.ac (VPN) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/vpn/vpn_ac.md) + - [VPN Autoconnect (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/vpn/vpn_autoconnect.md) ## Fundamental Tools - [Cockpit (recommended) (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/cockpit.md): simple browser-based management console/UI - [Chrony (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/chrony.md): clock synchronization -- [Borg Backups (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/borg.md): simple, effective backups +- [Borg Backups (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/borg.md): simple, effective backups - [Docker (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/docker.md): containerize your services -- [Let's Encrypt (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/lets_encrypt.md): SSL/TLS certificates for secure web communications +- [Let's Encrypt (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/acme_sh.md): SSL/TLS certificates for secure web communications - [Incron (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/incron.md): a way to watch the filesystem for configuration changes and restart services -- [Caddy (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/caddy.md): web server and reverse https proxy -- [Pi Hole (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/pi_hole.md): ad-blocking that blocks ads before they are loaded, can help conserve bandwidth and data transfer -- [Searx (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/searx.md): self-hosted meta search engine +- [Caddy (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/caddy.md): web server and reverse https proxy +- [Pi Hole (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/pi_hole.md): ad-blocking that blocks ads before they are loaded, can help conserve bandwidth and data transfer +- [Searx (link)](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/searx.md): self-hosted meta search engine ## Additional Services -- [Postgresl](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/postgres.md): database server, required for NextCloud, Gitea, Wallabag, and TT-RSS, and other services; helpful if you will be a heavy user of your Lollipop Cloud +- [Postgresl](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/postgres.md): database server, required for NextCloud, Gitea, Wallabag, and TT-RSS, and other services; helpful if you will be a heavy user of your Lollipop Cloud - [Monitoring](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/monitoring.md): basic system monitoring -- [NextCloud](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/nextcloud.md): file syncing, calendar syncing, contact syncing; an alternative to Dropbox and iCloud -- [Syncthing](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/syncthing.md): sync for large numbers or large sized files, an alternative to NextCloud +- [NextCloud](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/nextcloud.md): file syncing, calendar syncing, contact syncing; an alternative to Dropbox and iCloud +- [Syncthing](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/syncthing.md): sync for large numbers or large sized files, an alternative to NextCloud - Gitea (documentation needed): self-hosted git, alternative to GitHub and GitLab -- [Wallabag](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/wallabag.md): save websites for later, alternative to Read it Later and Pocket -- [TT-RSS](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/armbian/ttrss.md): self-hosted RSS reader, alternative to Google Reader +- [Wallabag](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/wallabag.md): save websites for later, alternative to Read it Later and Pocket +- [TT-RSS](https://git.lollipopcloud.solutions/lollipop-cloud/docs/src/branch/master/services/ttrss.md): self-hosted RSS reader, alternative to Google Reader diff --git a/advanced/virtualization.md b/advanced/virtualization.md new file mode 100644 index 0000000..162605b --- /dev/null +++ b/advanced/virtualization.md @@ -0,0 +1,208 @@ +# Virtualization + +Some notes on setting up Docker build hosts using emulation. Specifically using qemu to setup arm32v7 and arm64v8 environments for Docker builds. + +Handy if you need are using an SBC that's too 'small' for building searx/gogs but *can* run the final Docker image. + +## Inspiration / Further Reading + +- [https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/](https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/) +- [https://github.com/petrosagg/armv7hf-python-dockerhub/blob/master/Dockerfile](https://github.com/petrosagg/armv7hf-python-dockerhub/blob/master/Dockerfile) +- [https://github.com/ViViDboarder/docker-rpi-homebridge/blob/master/Dockerfile](https://github.com/ViViDboarder/docker-rpi-homebridge/blob/master/Dockerfile) + - This BREAKS when not cross building so you can't easily run docker build / up on a docker file w/ these in it +- [https://wiki.qemu.org/Documentation/Platforms/ARM](https://wiki.qemu.org/Documentation/Platforms/ARM) +- [https://translatedcode.wordpress.com/2016/11/03/installing-debian-on-qemus-32-bit-arm-virt-board/](https://translatedcode.wordpress.com/2016/11/03/installing-debian-on-qemus-32-bit-arm-virt-board/) +- [https://translatedcode.wordpress.com/2017/07/24/installing-debian-on-qemus-64-bit-arm-virt-board/](https://translatedcode.wordpress.com/2017/07/24/installing-debian-on-qemus-64-bit-arm-virt-board/) +- [https://www.raymii.org/s/articles/virt-install_introduction_and_copy_paste_distro_install_commands.html](https://www.raymii.org/s/articles/virt-install_introduction_and_copy_paste_distro_install_commands.html) +- [http://ftp.nl.debian.org/debian/dists/stretch/main/](http://ftp.nl.debian.org/debian/dists/stretch/main/) +- [https://www.collabora.com/news-and-blog/blog/2017/06/20/debian-armhf-vm-on-arm64-server/](https://www.collabora.com/news-and-blog/blog/2017/06/20/debian-armhf-vm-on-arm64-server/) +- [http://blog.system76.com/post/139138591598/howto-qemu-w-ubuntu-xenial-host-uefi-guest](http://blog.system76.com/post/139138591598/howto-qemu-w-ubuntu-xenial-host-uefi-guest) + +## Minor notes + +- Use arm64 for arm64v8 +- Use armhf for arm32v + +## Prep + +``` bash + +apt install qemu-system-arm libguestfs-tools qemu-efi qemu-efi-aarch64 qemu-efi-arm + +``` + +## Plain QEMU + +These processes will let you bootstrap a virtual machine for later import into libvirt management engines. + +### 64bit arm v8 + +``` bash + +# Probably should have this in a dedicated dir, no? +mkdir arm64v8 +cd arm64v8 +# Pull installer +wget http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-arm64/current/images/netboot/mini.iso +mv mini.iso net-install.iso +# Create storage for VM +qemu-img create -f qcow2 arm64v8-xenial.qcow2 16G +# Prep EFI boot +cp /usr/share/AAVMF/AAVMF_VARS.fd AAVMF_VARS.fd +# Install +# WARNING: YOU MUST TWEAK THE GRUB KERNEL CONSOLE LINE TO HAVE 'console=ttyAMA0' IMMEDIATELY FOLLOWING THE KERNEL IMAGE OR YOU WON'T GET OUTPUT +# NOTE: Use CTRL-H for backspace and CTRL-X to boot +# NOTE: virtio-blk-device and virtio-net-device are necessary as the mini.iso (renamed to net-install.iso) doesn't support PCI (fixed post-install) +qemu-system-aarch64 -M virt -m 2048 -smp 4 -cpu cortex-a53 \ + -drive if=pflash,format=raw,readonly,file=/usr/share/AAVMF/AAVMF_CODE.fd \ + -drive if=pflash,format=raw,file=AAVMF_VARS.fd \ + -cdrom net-install.iso \ + -drive if=none,file=arm64v8-xenial.qcow2,format=qcow2,id=hd \ + -device virtio-blk-device,drive=hd \ + -netdev user,id=arm0 -device virtio-net-device,netdev=arm0 \ + -nographic -no-reboot +# Backup base install (just in case) +cp arm64v8-xenial.qcow2 arm64v8-xenial-base-install.qcow2 +# Run the VM with the necessary parms/infos +qemu-system-aarch64 -M virt -m 2048 -smp 4 -cpu cortex-a53 \ + -drive if=pflash,format=raw,readonly,file=/usr/share/AAVMF/AAVMF_CODE.fd \ + -drive if=pflash,format=raw,file=AAVMF_VARS.fd \ + -drive if=none,file=arm64v8-xenial.qcow2,format=qcow2,id=hd \ + -device virtio-blk-device,drive=hd \ + -netdev user,id=arm0 -device virtio-net-device,netdev=arm0 \ + -nographic -no-reboot + +``` + +### 32bit arm v7 + +``` bash + +# Probably should have this in a dedicated dir, no? +mkdir arm32v7 +cd arm32v7 +# Get netbook kernel/initrd +wget -O installer-vmlinuz http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-armhf/current/images/generic-lpae/netboot/vmlinuz +wget -O installer-initrd.gz http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-armhf/current/images/generic-lpae/netboot/initrd.gz +# Create storage for VM +qemu-img create -f qcow2 arm32v7-xenial.qcow2 16G +# Prep EFI boot +cp /usr/share/AAVMF/AAVMF32_VARS.fd AAVMF32_VARS.fd +# Install +qemu-system-arm -M virt -m 2048 -smp 4 -cpu cortex-a15 \ + -kernel installer-vmlinuz \ + -initrd installer-initrd.gz \ + -drive if=none,file=arm32v7-xenial.qcow2,format=qcow2,id=hd \ + -device virtio-blk-pci,drive=hd \ + -netdev user,id=arm0 -device virtio-net-pci,netdev=arm0 \ + -nographic -no-reboot +# Backup base install (just in case) +cp arm32v7-xenial.qcow2 arm32v7-xenial-base-install.qcow2 +# Copy vmlinuz/initrd from root filesystem for final boot +virt-filesystems -a arm32v7-xenial.qcow2 +virt-ls -a arm32v7-xenial.qcow2 /boot/ +virt-copy-out -a arm32v7-xenial.qcow2 /boot/vmlinuz-4.4.0-127-generic-lpae /boot/initrd.img-4.4.0-127-generic-lpae . +# Boot! +qemu-system-arm -M virt -m 2048 -smp 4 -cpu cortex-a15 \ + -kernel vmlinuz-4.4.0-127-generic-lpae \ + -initrd initrd.img-4.4.0-127-generic-lpae \ + -drive if=none,file=arm32v7-xenial.qcow2,format=qcow2,id=hd \ + -device virtio-blk-pci,drive=hd \ + -netdev user,id=arm0 -device virtio-net-pci,netdev=arm0 \ + -nographic -no-reboot + +``` + +## virsh/virt-install + +This section is very much a work in progress. Based on further research / reading you'll need to bootstrap the VM with raw qemu and then import to virsh. + +``` bash + +virt-install --name ubuntu-16.04-arm64v8 \ + --import \ + --virt-type qemu \ + --arch aarch64 \ + --machine virt \ + --memory 2048 \ + --vcpus 4 \ + --noacpi \ + --os-type linux \ + --os-variant ubuntu16.04 \ + --disk path=template-ubuntu-16.04-arm64v8.qcow2,size=25,bus=virtio \ + --network default \ + --graphics none \ + +``` + +## Useful virsh commands + +For post install usage + +``` bash + +virsh list --all # List all VMs +virsh dumpxml --domain VM_NAME # Dump the XML for the VM +virsh destroy --domain VM_NAME # Force shutdown VM +virsh undefine --domain VM_NAME # Remove/Delete VM +virsh console VM_NAME +virsh edit VM_NAME +rm DISK_IMAGE_PATH + +``` + +## First Boot + +``` bash + +vim /etc/default/grub +update-grub +apt update && apt install sudo vim tmux screen wget curl htop git nano rsync iotop nload unzip +ufw allow ssh +ufw enable + +``` + +## rclone + +``` bash + +curl https://rclone.org/install.sh | sudo bash + +``` + +## Docker + +``` bash + +apt remove docker docker-engine docker.io +apt install apt-transport-https ca-certificates curl gnupg2 software-properties-common + +echo "deb [arch=arm64] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list + +echo "deb [arch=armhf] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list + +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - +apt update +apt install docker-ce python3 python3-pip +pip3 install docker-compose +poweroff + +``` + +## Export Docker Container Images + +``` bash + +for repo in `docker image ls --format "{{.Repository}}"`; do + outfile=`echo $repo | sed 's+/+_+g'` + echo "$repo -> $outfile" + docker save -o ${outfile}.tar $repo +done + +``` + diff --git a/armbian/vpn/pia.md b/armbian/vpn/pia.md new file mode 100644 index 0000000..6ba4231 --- /dev/null +++ b/armbian/vpn/pia.md @@ -0,0 +1,237 @@ +# Private Internet Access (PIA) + +```TODO : INCOMPLETE``` + +This is a **VERY** advanced topic with some creative tricks to simplify getting the config added to NetworkManager. You're on your own. + +The author *strongly* recommends reading through this and adapting to other services well ahead of any real need(s). + +## Inspiration / Sources + +- [http://blog.deadlypenguin.com/blog/2017/04/24/vpn-auto-connect-command-line/](http://blog.deadlypenguin.com/blog/2017/04/24/vpn-auto-connect-command-line/) +- [https://forums.linuxmint.com/viewtopic.php?t=97187](https://forums.linuxmint.com/viewtopic.php?t=97187) +- [https://unix.stackexchange.com/questions/301845/scripting-a-way-to-quickly-import-ovpn-files-to-networkmanager-on-ubuntu](https://unix.stackexchange.com/questions/301845/scripting-a-way-to-quickly-import-ovpn-files-to-networkmanager-on-ubuntu) +- [https://www.privateinternetaccess.com/pages/client-support/](https://www.privateinternetaccess.com/pages/client-support/) + +## Pep Work + +Download PIA OpenVPN advanced configs, drop them in a good place on the filesystem and script import. + +All files can be found Under ```Advanced OpenVPN SSL Restrictive Configurations``` on the main PIA website. + +### Download PIA OpenVPN Template Files + +``` bash + +mkdir -p /etc/pia +cd /etc/pia +mkdir openvpn-strong +cd openvpn-strong +wget https://www.privateinternetaccess.com/openvpn/openvpn-strong.zip +unzip openvpn-strong.zip +cd .. +mkdir openvpn-strong-tcp +cd openvpn-strong-tcp +wget https://www.privateinternetaccess.com/openvpn/openvpn-strong-tcp.zip +unzip openvpn-strong-tcp.zip +cd .. + +``` + +### Add OpenVPN To Network Manager + +``` bash + +# NetworkManager Gnome is *required* to get a missing library on xenial +# OMIT Gnome item if you can get away with it... + +apt update +apt install network-manager-openvpn network-manager-openvpn-gnome + +``` + +### Cleanup Files for Import + +The cipher lines in the templates provided by PIA won't work 100% correctly on import. These commands will clean them up for import. + +``` bash + +cd /etc/pia +cd openvpn-strong +sed -i 's/aes-256-cbc/AES-256-CBC/g' *.ovpn +sed -i 's/sha256/SHA256/g' *.ovpn +sed -i 's_crl.rsa.4096.pem_/etc/pia/openvpn-strong/crl.rsa.4096.pem_g' *.ovpn +sed -i 's_ca.rsa.4096.crt_/etc/pia/openvpn-strong/ca.rsa.4096.crt_g' *.ovpn +cd .. +cd openvpn-strong-tcp +sed -i 's/aes-256-cbc/AES-256-CBC/g' *.ovpn +sed -i 's/sha256/SHA256/g' *.ovpn +sed -i 's_crl.rsa.4096.pem_/etc/pia/openvpn-strong-tcp/crl.rsa.4096.pem_g' *.ovpn +sed -i 's_ca.rsa.4096.crt_/etc/pia/openvpn-strong-tcp/ca.rsa.4096.crt_g' *.ovpn +cd .. + +``` + +## Import / Setup + +### Manual Import + +Import the cleaned up, provided OpenVPN profiles. This will require intervention on your part after the profile is created in NetworkManager to ensure it can connect properly. + +*Note: this assumes UDP profiles, tweak for the ```openvpn-strong-tcp``` directory if desired or necessary* + +#### Initial Import / Edit + +``` bash + +nmcli connection import type openvpn file /etc/pia/openvpn-strong/[con_name].ovpn +nmcli con show +vim /etc/NetworkManager/system-connections/[con_name] + +``` +#### Adjustments/Necessary Verification + +``` bash + +id=PIA - [con_name] + +[vpn] +auth=SHA256 +cipher=AES-256-CBC +password-flags=0 +user-name=[your_username] + +[vpn-secrets] +password=[your_password] + +``` + +#### Update Profiles in NetworkManager + +``` bash + +# Reload For Changes +nmcli connection reload [uuid_from_above] + +# Connect +nmcli con up [uuid_from_above] + +``` + +### Scripted Import + +Some automated imports based on how the ```ovpn``` files are normally imported via ```nmcli``` + +*Note: this assumes UDP profiles, you'll need to tweak this as you see fit for TCP* + +#### Install ```uuidgen``` + +``` bash + +apt update +apt install uuid-runtime + +``` + +#### Create List of VPN Endpoints + +``` bash + +cat > /etc/pia/server_list.txt << EOF +us-west.privateinternetaccess.com=PIA - USA (West) +us-east.privateinternetaccess.com=PIA - USA (East) +us-midwest.privateinternetaccess.com=PIA - USA (Midwest) +aus.privateinternetaccess.com=PIA - Australia (Sydney) +austria.privateinternetaccess.com=PIA - Austria +belgium.privateinternetaccess.com=PIA - Belgium +ca-toronto.privateinternetaccess.com=PIA - Canada (Toronto) (East) +ca-vancouver.privateinternetaccess.com=PIA - Canada (Vancouver) (West) +fi.privateinternetaccess.com=PIA - Finland +france.privateinternetaccess.com=PIA - France +germany.privateinternetaccess.com=PIA - Germany +hk.privateinternetaccess.com=PIA - Hong Kong +in.privateinternetaccess.com=PIA - India +japan.privateinternetaccess.com=PIA - Japan +mexico.privateinternetaccess.com=PIA - Mexico +nl.privateinternetaccess.com=PIA - Netherlands +no.privateinternetaccess.com=PIA - Norway +sg.privateinternetaccess.com=PIA - Singapore +spain.privateinternetaccess.com=PIA - Spain +sweden.privateinternetaccess.com=PIA - Sweden +swiss.privateinternetaccess.com=PIA - Switzerland +turkey.privateinternetaccess.com=PIA - Turkey +uk-london.privateinternetaccess.com=PIA - UK (London) +brazil.privateinternetaccess.com=PIA - Brazil +EOF + +``` + +#### Setup NeworkManager Profiles + +Some fancy bash tricks to get the full list of NetworkManager PIA connections imported in one copy/paste. + +*Note: You'll need to fill in ```[your_username]``` and ```[your_password]``` before running this block of commands. + +``` bash + +export PIA_USER="[your_username]" +export PIA_PASSWORD="[your_password]" +while read line; +do +desc=$(echo $line | cut -f2 -d'=') +dns=$(echo $line | cut -f1 -d'=') +file="/etc/NetworkManager/system-connections/$desc" + +cat > "$file" < /etc/autovpn/default + +``` + +## Setup Docker networks to exclude + +``` bash + +# Get network names of ALL docker connections +nmcli con show +echo "[name1],[name2]" > /etc/autovpn/exclude + +``` + +## Setup script that will auto activate VPN + +This will auto-activate whenever NetworkManager says that the network is in a state of ```Connected-Global```. It may take a few minutes to get the whole network online/activated. + +Put the below script at ```/etc/autovpn/autovpn.py``` + +``` python + +#!/usr/bin/env python3 +""" +Copyright 2011 Domen Kozar. All rights reserved. + +Redistribution and use in source and binary forms, with or without modification, are +permitted provided that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright notice, this list of + conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright notice, this list + of conditions and the following disclaimer in the documentation and/or other materials + provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY DOMEN KOZAR ''AS IS'' AND ANY EXPRESS OR IMPLIED +WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DOMEN KOZAR OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF +ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +The views and conclusions contained in the software and documentation are those of the +authors and should not be interpreted as representing official policies, either expressed +or implied, of DOMEN KOZAR. + +USAGE +===== + +1) clone gist somewhere (eg. /home/user/autovpn/) +2) add to /etc/rc.local: python /home/user/autovpn/autovpn.py "myvpn" 'Auto homenetwork,Auto worknetwork' >> /var/log/autovpn.log& +3) reboot :-) + +CHANGELOG +========= + +0.2 (28.01.2012) +---------------- + +* feature: use logging module +* bug: script would fail if there was no active connection + +0.1 (01.01.2012) +---------------- + +* bug: compatible with NM 0.9, dropped support for 0.8 +* feature: specify networks that vpn is not autoconnected + +KNOWN ISSUES +============ + +* it will always use first active network connection + +""" +import sys +import logging + +from dbus.mainloop.glib import DBusGMainLoop +import dbus +from gi.repository import GObject as gobject + + +logger = logging.getLogger(__name__) +logging.basicConfig( + level=logging.INFO, + filename='/var/log/autovpn.log', + filemode='a', +) + + +class AutoVPN(object): + """Solves two jobs, tested with NetworkManager 0.9.x: + + * if VPN connection is not disconnected by user himself, reconnect (configurable max_attempts) + * on new active network connection, activate VPN + + :param vpn_name: Name of VPN connection that will be used for autovpn + :param ignore_networks: Comma separated network names in NM that will not force VPN usage + :param max_attempts: Maximum number of attempts of reconnection VPN session on failures + :param delay: Miliseconds to wait before reconnecting VPN + + """ + + def __init__(self, vpn_name, ignore_networks='', max_attempts=5, delay=5000): + self.vpn_name = vpn_name + self.max_attempts = max_attempts + self.delay = delay + self.failed_attempts = 0 + self.bus = dbus.SystemBus() + self.ignore_networks = ignore_networks.split(',') + print(self.ignore_networks) + self.activate_vpn() # Auto connect at startup (Listen for StateChanged going forward) + self.get_network_manager().connect_to_signal("StateChanged", self.onNetworkStateChanged) + logger.info("Maintaining connection for %s, reattempting up to %d times with %d ms between retries", vpn_name, max_attempts, delay) + + def onNetworkStateChanged(self, state): + """Handles network status changes and activates the VPN on established connection.""" + logger.debug("Network state changed: %d", state) + if state == 70: + self.activate_vpn() + + def onVpnStateChanged(self, state, reason): + """Handles different VPN status changes and eventually reconnects the VPN.""" + # vpn connected or user disconnected manually? + if state == 5 or (state == 7 and reason == 2): + self.failed_attempts = 0 + if state == 5: + logger.info("VPN %s connected", self.vpn_name) + else: + logger.info("User disconnected manually") + return + # connection failed or unknown? + elif state in [6, 7]: + # reconnect if we haven't reached max_attempts + if not self.max_attempts or self.failed_attempts < self.max_attempts: + logger.info("Connection failed, attempting to reconnect") + self.failed_attempts += 1 + gobject.timeout_add(self.delay, self.activate_vpn) + else: + logger.info("Connection failed, exceeded %d max attempts.", self.max_attempts) + self.failed_attempts = 0 + + def get_network_manager(self): + """Gets the network manager dbus interface.""" + logger.debug("Getting NetworkManager DBUS interface") + proxy = self.bus.get_object('org.freedesktop.NetworkManager', '/org/freedesktop/NetworkManager') + return dbus.Interface(proxy, 'org.freedesktop.NetworkManager') + + def get_vpn_interface(self, name): + 'Gets the VPN connection interface with the specified name.' + logger.debug("Getting %s VPN connection DBUS interface", name) + proxy = self.bus.get_object('org.freedesktop.NetworkManager', '/org/freedesktop/NetworkManager/Settings') + iface = dbus.Interface(proxy, 'org.freedesktop.NetworkManager.Settings') + connections = iface.ListConnections() + for connection in connections: + proxy = self.bus.get_object('org.freedesktop.NetworkManager', connection) + iface = dbus.Interface(proxy, 'org.freedesktop.NetworkManager.Settings.Connection') + con_settings = iface.GetSettings()['connection'] + if con_settings['type'] == 'vpn' and con_settings['uuid'] == name: + logger.debug("Got %s interface", name) + return iface + logger.error("Unable to acquire %s VPN interface. Does it exist?", name) + return None + + def get_active_connection(self): + """Gets the dbus interface of the first active + network connection or returns None. + """ + logger.debug("Getting active network connection") + proxy = self.bus.get_object('org.freedesktop.NetworkManager', '/org/freedesktop/NetworkManager') + iface = dbus.Interface(proxy, 'org.freedesktop.DBus.Properties') + active_connections = iface.Get('org.freedesktop.NetworkManager', 'ActiveConnections') + if len(active_connections) == 0: + logger.info("No active connections found") + return None + logger.info("Found %d active connection(s)", len(active_connections)) + return active_connections + + def activate_vpn(self): + """Activates the vpn connection.""" + logger.info("Activating %s VPN connection", self.vpn_name) + vpn_con = self.get_vpn_interface(self.vpn_name) + active_cons = self.get_active_connection() + if active_cons is None: + print('active cons is None') + return + + inet_con = None + for active_con in active_cons: + print(active_con) + # check if we have to ignore vpn + proxy = self.bus.get_object('org.freedesktop.NetworkManager', active_con) + con = dbus.Interface(proxy, 'org.freedesktop.DBus.Properties').Get('org.freedesktop.NetworkManager.Connection.Active', 'Connection') + proxy = self.bus.get_object('org.freedesktop.NetworkManager', con) + settings = dbus.Interface(proxy, 'org.freedesktop.NetworkManager.Settings.Connection').GetSettings() + if settings['connection']['id'] not in self.ignore_networks: + print('found valid con') + inet_con = active_con + break + + if inet_con is None: + print('no valid con') + logger.info("Ignored network connections based on settings") + return + + # activate vpn and watch for reconnects + if vpn_con and inet_con: + print('activating vpn') + print(inet_con) + try: + new_con = self.get_network_manager().ActivateConnection( + vpn_con, + dbus.ObjectPath("/"), + inet_con, + ) + proxy = self.bus.get_object('org.freedesktop.NetworkManager', new_con) + iface = dbus.Interface(proxy, 'org.freedesktop.NetworkManager.VPN.Connection') + iface.connect_to_signal('VpnStateChanged', self.onVpnStateChanged) + logger.info("VPN %s should be active (soon)", self.vpn_name) + except dbus.exceptions.DBusException: + # Ignore dbus connections (in case VPN already active when this script runs) + # TODO: Do this handling better; maybe check active/inactive status above and bail if already active? + print('dbus.exceptions.DBusException') + pass + +if __name__ == '__main__': + if len(sys.argv) < 2: + print('usage: autovpn VPN_CONNECTION_NAME ') + print('-> activates vpn if any network connection is active') + print('-> and reconnects VPN on failure') + sys.exit(0) + + # set up the main loop + DBusGMainLoop(set_as_default=True) + loop = gobject.MainLoop() + # TODO: argparse + if len(sys.argv) > 2: + AutoVPN(sys.argv[1], sys.argv[2]) + else: + AutoVPN(sys.argv[1]) + loop.run() + +``` + +## Setup Script + +``` bash + +# Set permissions as required by NetworkManager +chown root:root /etc/autovpn/autovpn.py +chmod 0700 /etc/autovpn/autovpn.py + +``` + +## Setup the script as a systemd service + +``` bash + +cat > /etc/systemd/system/autovpn.service < storage > removable storage > format. Source: [SD card formatting (link).](https://raspberrypi.stackexchange.com/questions/1446/how-can-i-reformat-my-sd-card-to-use-it-normally-again#comment81498_1446) + +#### Once you've formatted your SD card: + +If you don't already have one, you will also need an application/program to properly prepare your formatted SD card. [Etcher (link)](https://etcher.io/) works on most computers. To use Etcher: Insert the SD card into your computer (using an built-in reader or an external USB reader), load Etcher, select your newly extracted Armbian .img file, select the SD card you want to set up, and click Flash! + +When Etcher is finished, you're ready to insert your Armbian SD card into your Orange Pi. Insert your HDMI cable or serial console, and keyboard or ethernet if you are using it, and then plug in the power cord. The Orange Pi should boot automatically. + +If your board does not boot (the screen is blank or you cannot connect via SSH), but the power light is on and your SD card is properly inserted into the Orange Pi (make sure it clicks into place!) your SD card may not have flashed properly. Repeat the process for formatting the SD card and running Etcher, and try again. + +# Your First Time Booting Armbian + +Now you're reading to turn on your Lollipop device for the first time! + +If you're not using a keyboard, connect your Orange Pi (or the board you're using) with an ethernet cable to your local router and use your development computer to load Terminal (if using a Mac) and run `ssh root@192.168.1.x` (your board's IP address! make sure to use the root@IP_address format or else Terminal will tell the Orange Pi that you're trying to log in with your Mac's username) OR use [PuTTY (link)](https://putty.org/) to open an SSH connection to your Orange Pi. You may need to log into your router's settings to [find the IP address of your Orange Pi (link)](https://www.howtogeek.com/204057/how-to-see-who%E2%80%99s-connected-to-your-wi-fi-network/). + +Both keyboard and ethernet users will continue on the same path here: + +![First login screen](/screenshots/first-login.png) + +At first boot, you will be prompted to log in with the default login (root) and password (1234), and then prompted to change your root password. When you set your root password, make it a good one and don't forget it! Anyone with root access to your computer will have access to everything on that computer, including the ability to make malicious changes. + +Next, you will be prompted to create a new user account and password. This account will have [sudo (link)](https://en.wikipedia.org/wiki/Sudo) privileges, which is just as powerful (and dangerous) as root access, so you'll need another secure password. It is good practice to never login as root, so you'll be using this new user account for everything going forward. + +Once you've created an account, the desktop environment will load, and it's time to start setting things up! + +NOTE: If you are connected via SSH (running without a display or "headless"), you can also install [VNC Viewer (link)](https://www.realvnc.com/) on your development computer, and [set up VNC (Virtual Network Computing) on your Orange Pi (link)](https://forum.pine64.org/showthread.php?tid=794), so you can connect virtually without having to deal with a separate keyboard/monitor/etc setup. This is something you may use many times in the future, so it's worth taking the time to set it up, even if you're currently using a dedicated keyboard and display for your Lollipop. [More information about VNC. (link)](https://en.wikipedia.org/wiki/Virtual_Network_Computing) + +![The view with VNC Viewer](/screenshots/vnc-viewer.png) + +# Base Setup + +At this point, you might be ready to jump into the [Lollipop base setup code (link)](https://git.lollipopcloud.solutions/lollipop-cloud/lolipop_lan_cloud/src/branch/master/docs/armbian/base_setup.md). If you need a little more detail, continue following along here instead. + +If you are sticking with the command line via SSH, these steps will hopefully be straightforward. If you are using your new Lollipop with the desktop environment, you will have to open up a terminal window. + +It's good practice to run `sudo apt update` at the command line to make sure everything is up to date, and you may need to run `sudo apt upgrade` as well (the command line will tell you if there are upgrades but it won't hurt to run it either way). After updating and upgrading, run `systemctl reboot`, give the board a minute or two to reboot, and then reconnect via SSH. + +__Note:__ if you get an error like `perl: warning: Falling back to a fallback locale ("en_US.UTF-8"). +locale: Cannot set LC_CTYPE to default locale: No such file or directory +locale: Cannot set LC_ALL to default locale: No such file or directory`, try the following command: `export LC_ALL="en_US.UTF-8"`. If you continue to get this error, you can edit the /etc/ssh/ssh_config file (see the next paragraph for more detail about file editing, but you will have to do `sudo nano /etc/ssh/ssh_config`) and comment out (add `#` to) the `SendEnv LANG LC_*` line. + + +You will have to edit some files. Nano is a simple built-in text editor, but feel free to use your text editor of choice. Examples here will use Nano. + +You will be disabling auto-updates in order to save bandwidth and time, and to prevent data overages for those with data caps. + +First file to edit: +`sudo nano /etc/apt/apt.conf.d/02periodic` + and change the line ```APT::Periodic::Enable "1";``` to ```APT::Periodic::Enable "0";``` + + Then enter command `control-x` (to exit) and choose `y` to save changes. Hit enter to keep the file name the same, and now you're reading to move on to the next file. + +Second file to edit: `sudo nano /etc/apt/apt.conf.d/20auto-upgrades` and change the line ```APT::Periodic::Update-Package-Lists "1";``` to ```APT::Periodic::Update-Package-Lists "0";``` + +AND change ```APT::Periodic::Unattended-Upgrade "1";``` to ```APT::Periodic::Unattended-Upgrade "0";``` + +(remember to `ctrl-x` and `y` to save your changes). + +Third file to edit: `sudo nano /etc/ssh/sshd_config` and make sure the following is set: `PermitRootLogin no`. This is a longer file, so enter `ctrl-w` to search for `PermitRootLogin`. You may have to `ctrl-w` and `enter` again to find the line you're looking for. Change `yes` to `no` if necessary and then save your changes (`ctrl-x` and `y`). + +Now restart the service with `systemctl restart sshd`. + +# Congratulations! You've installed an entire operating system and completed the base setup! Time to customize your Lollipop! + +Now you're ready to head back to the [README (link)](docs/README.md) and get started on the networking section. + +**Closing suggestion:** It is highly recommended that once you set up networking on your new Lollipop Cloud, that you follow the README's suggestion to install Cockpit. After you install Cockpit, you will be able to configure the rest of your desired Lollipop Cloud features through a web browser on your development (or home) computer. + +_Check and contribute to [the issue tracker (link)](https://kanban.lollipopcloud.solutions/project/admin-lollipop-documentation/kanban) for errors, typos, clarifications, and omissions to improve this documentation. Thanks!_ diff --git a/for-beginners/how-to-gitea.md b/for-beginners/how-to-gitea.md new file mode 100644 index 0000000..cadb4b4 --- /dev/null +++ b/for-beginners/how-to-gitea.md @@ -0,0 +1,118 @@ +# You don't have to learn git or Gitea to contribute... +If you would like to walk through the git contribution process to submit changes, skip this section and move down to the [cheatsheat](#"Gitea cheat sheet for beginners and the forgetful"). But if you have no interest in expending that much energy, you can skip git entirely and submit a detailed suggestion to our [kanban board (link)](https://kanban.lollipopcloud.solutions/). + +# Gitea cheat sheet for beginners and the forgetful +Below is the basic Gitea workflow, to help you with your contributions. But do not hesitate to reach out to any of the organizers for help! We're happy to walk you through any or all of this, no matter what your experience level. If you've never even heard of Gitea before, we'd love to help you submit your first contribution! + +This document gets wordy because Git is not as welcoming to beginners as we wish it was, but you can do it! And you're not alone! We are just a message or comment away. There are even more resources in the [Bibliography](#Bibliography) at the end, if you want to dive deeper. You may even want to skip this document entirely at first, and see if [Your First Pull Request (link)](https://joshuahunter.com/posts/your-first-pull-request/) more suited to your needs. + +If you need more Git, check out the [Git Book (link)](https://git-scm.com/book/en/v2). + +# Basic Steps For Contributing +These are the general steps, but if you need more information, keep reading. + +* Fork the repository (just once) from [Lollipop Cloud project page (link)](https://git.lollipopcloud.solutions/) +* Clone your fork on your own computer (just once) +* Create a branch for your changes +* Edit, make changes and contributions +* Push those changes to your branch +* Open a merge/pull request in Gitea +* Wait for project coordinator to provide feedback or approve the merge/pull request + +# And now... what to do if you've never done this before + +### Log in to Gitea +If you haven't already, create an account on the [Lollipop Cloud project page (link)](https://git.lollipopcloud.solutions/) and log in. + +### Fork it! (copy the project to your Gitea account) +View the repository you would like to work on (docs or website), and click `Fork` in the upper right corner. You will only have to fork once! + +A fork is an exact copy of the project as it existed at the exact moment you forked it. So when the project updates in the future, you will have to update your code too, but we will talk about that later. + +At this point, you will need a command line client like [PuTTY (link)](https://putty.org/) for Windows users, or the built-in Terminal for MacOS users. + +### Go to (or create) a directory where you'd like your copy of the project to live, and open it. +In PuTTY or Terminal, enter each of the following commands, where "lollipop_project" is the name of directory where you would like to store the clone. + +`mkdir /path/to/lollipop_project` + +`cd /path/to/lollipop_project` + +### Clone (create a local copy) of the project files +View your newly created fork in your web browser and copy the https:// address in the box on the right side of the page. +In your terminal program, enter `git clone https://git.lollipopcloud.solutions/your-username/forked-repository`, with the correct URL. +You are downloading all the project files to your computer! Good work! + +### View your cloned project on your computer +Git will install the project in a new folder. Use the command `ls` to see the name of the new project folder if you are unsure. Open this new folder with +`cd cloned_directory_name` to see what was installed. + +### Add the upstream path (Do this step only once!) +If the master (where you forked FROM) changes, you will have to update your local clone. You will only have to do this step ONE time! + +From your `cloned_directory_name` directory, do this: + +`git remote add upstream https://git.lollipopcloud.soltuions/lollipop_cloud/original_repository.git` (with the original repository URL). + +### Create a branch +The `git checkout` command at the end will create a branch, but this is a good practice for each time you want to create a new branch for contributing: + +`git checkout master` + +`git fetch upstream` + +`git rebase upstream/master` + +`git checkout -b new_branch_name` + +Now the changes you make will be associated with `new_branch_name`. + +### Edit your contributions +Now that the files live locally on your computer, and you have a branch set up for your new contributions, you can edit the files with your favorite editor. Stick with the command line (Vim, Emacs, Nano), or use a text editor like [Atom (link)](https://atom.io/). Be sure to save your changes. + +### Add your changed files to git, to prepare them for uploading +If you added new files to the project, in your terminal you will have to `git add edited-file-name` for each of those new files. + +### Commit your changes +When you're ready, go back to the command line and commit your changes with a short but descriptive comment. +`git commit -m "Updated List of Anthropomorphic Potatoes"` + +### Push your changes to the branch in your fork +When you're really ready, "push" those changes to the original source, which is a request to accept and merge your contributions with the rest of the project. **Note:** A push request and a merge request are the same thing. +`git push -u origin new_branch_name` + +### Create a pull request (AKA merge request) + +Go back to your browser and your forked repository `https://git.lollipopcloud.solutions/your-username/forked-repository` and go to Pull Requests and choose `New Pull Request`. + +There will be two drop-down menus that will probably say `base: master` and `compare: master`. The "base" fork is the original project, NOT your forked version. Under the `compare` menu, choose your `new_branch_name`. Enter a short title for the changes, a description to help reviewers understand your contributions, and click the big `Submit Pull Request` button below your text. + +## Congratulations! You've submitted your first pull request! 🎉 + +The developers will review your changes and work with you on next steps. + +# Troubleshooting + +## What to do when "upstream" changes... +"Upstream" is the name of the original code (not your fork). When it gets updated after you clone or fork, you will have to request a copy of those changes to stay up-to-date. + +### Fetch the changes since you first forked or last fetched changes +In order to be sure you're working off the latest copy of the project, you can fetch the latest changes with `git fetch upstream`. (Note: this can only be done at the command line, not the user interface/URL.) + +Now you have the latest copy of the project in your local directory. Your new changes from this point forward will be stored in a local branch called `upstream/master`. + +Make your edits! Then you can go back to the section [about adding your changed files to git](#Add your changed files to git, to prepare them for uploading). + + +# Bibliography + +* [Atom text editor (link)](https://atom.io/) +* [Git Book (link)](https://git-scm.com/book/en/v2) +* [Git Cheat Sheet (link)](https://github.com/mikeizbicki/ucr-cs100/blob/2015winter/textbook/cheatsheets/git-cheatsheet.md): May be helpful for setting up Git from scratch. +* [Git Cheat Sheet - Advanced (link)](https://github.com/mikeizbicki/ucr-cs100/tree/2015winter/textbook/tools/git/advanced-git) +* [Git Game (link)](https://www.git-game.com/): get serious about Git by gameifying your learning. +* [Oh shit, git! (link)](http://ohshitgit.com/): how to fix a few Git mistakes (we all make them!) +* [PuTTY SSH and telnet client (link)](https://putty.org/) +* [Syncing a fork (link)](https://help.github.com/articles/syncing-a-fork/) +* [Vim Cheat Sheet (link)](http://vimsheet.com/) +* [Your First Pull Request (link)](https://joshuahunter.com/posts/your-first-pull-request/) diff --git a/hardware/README.md b/hardware/README.md index ce04040..bf2b2b1 100644 --- a/hardware/README.md +++ b/hardware/README.md @@ -1,3 +1,121 @@ -# Hardware +# Hardware Notes -Setup additional hardware with an SBC \ No newline at end of file +Some thoughts and suggestions about Lollipop hardware options. + +## Official Boards + +After much testing, discussion, success, failure, and inspiration... we are happy to announce the boards we will officially support! + +We've selected 4 different boards as our primary targets for the Lollipop Cloud project. Our team members will be working with these boards heavily and will prioritize supporting them. The boards were selected for their price ($25-$100 USD), specifications, and ease-of-use, keeping in mind our goal of making self-hosted clouds an accessible reality. + +### Orange Pi PC Plus + +The [Orange Pi PC Plus (product details / link)](http://www.orangepi.org/orangepipcplus/) is a reasonably priced arm32v7 board. This board can be purchased as a full set that includes a board, case, and power supply. It's got enough RAM, CPU, and more to run a full self-hosted cloud for an individual, family, and maybe a little more. + +It has an on-board WiFi chip and on-board ethernet so networking won't be a problem. It has 3 USB ports for additional WiFi adapters or USB disks. It also has a full HDMI connector so you can setup using a USB keyboard and TV/monitor/etc. + +This is our recommended board for deploying a Lollipop Cloud, especially for our beginner users. + +### Orange Pi Plus 2e + +The [Orange Pi Plus 2e (product details / link)](http://www.orangepi.org/orangepiplus2e/) is an arm64v8 board. It packs 2Gb of RAM, 4 core CPU, lots of expansion options, built-in WiFi, built-in ethernet, and more. + +This board can run a self-hosted cloud for an individual, family, or small group. If you're looking for the "big board," this is it. + +### O-Droid HC2 + +The [O-Droid HC2 (product details / link)](https://www.hardkernel.com/main/products/prdt_info.php?g_code=G151505170472) is an expensive board that packs the most computing power of our target boards. We are recommending this board for those who want to build their own "home brew" Synology NAS. + +While not the cheapest option, this should be a reasonable replacement for entry level Synology NAS boxes if you're looking to self-host a NAS + Cloud without using proprietary software. + +### Orange Pi Zero Plus + +The [Orange Pi Zero Plus (product details / link)](http://www.orangepi.org/OrangePiZeroPlus/) is a small, inexpensive arm64v8 board. This board is a great starting point and can be used as a little pocket router with ad blocking, VPN, and more. This is best used as a WiFi hot spot device. It can also function as a very basic cloud but does *NOT* have quite enough RAM to function as a full cloud. + +This is a great choice if you're looking for something pocket sized and/or minimal. + +## Expansion Options + +### USB Storage + +If you're going to run services like NextCloud, Syncthing and/or Samba you will want to consider what kind of USB disk to use. I'd recommend a [Sandisk Cruzer Fit (link)](https://www.sandisk.com/home/usb-flash/cruzer-fit) or [Sandisk Ultra Fit (link)](https://www.sandisk.com/home/usb-flash/ultra-fit-usb) or similar, low power flash drive. Full hard disks that aren't SSD's tend to be power hungry and can cause problems with SBCs. + +### WiFi + +- The author has had decent luck with the [ASUS USB-N13 (link)](https://www.asus.com/us/Networking/USB-N13/) adapter in client mode. Be minful of your 2.4ghz channel setup. If this adapter and your AP share a channel in 2.4ghz this adapter *will* drop packets and/or suffer disconnections + - Sometimes a reboot is necessary to get authentication to work properly after initial setup + - The author has **NOT** tested AP mode +- The author has had very good luck with the [ASUS USB-N53 (link)](https://www.asus.com/us/Networking/USBN53/) in AP mode. *Both* 2.4ghz and 5ghz can used for AP mode at the same time. + - The author has **NOT** tested client mode +- The author has had decent luck with the TP-Link N150 USB WiFi adapter in client mode +- The author could not get the TP-Link N300 USB WiFi adapter to work in client mode + +### WiFi Antennas + +- [https://www.mouser.com/ProductDetail/?qs=%2fv8iy7V9uiwj65CKT%2f%252b6tQ%3d%3d](https://www.mouser.com/ProductDetail/?qs=%2fv8iy7V9uiwj65CKT%2f%252b6tQ%3d%3d) +- [https://www.mouser.com/ProductDetail/?qs=G9o9YCnxvoZVoyw0A06Ktg%3d%3d](https://www.mouser.com/ProductDetail/?qs=G9o9YCnxvoZVoyw0A06Ktg%3d%3d) +- [https://www.mouser.com/ProductDetail/?qs=WUa1z%2fNV9%252b2lzv2ZS%2f50GQ%3d%3d](https://www.mouser.com/ProductDetail/?qs=WUa1z%2fNV9%252b2lzv2ZS%2f50GQ%3d%3d) +- [https://www.mouser.com/ProductDetail/?qs=RuW%2fu%252bNMQmv6yDroBT8RNQ%3d%3d](https://www.mouser.com/ProductDetail/?qs=RuW%2fu%252bNMQmv6yDroBT8RNQ%3d%3d) +- [https://www.mouser.com/ProductDetail/Antenova/SR4W030-100?qs=sGAEpiMZZMuBTKBKvsBmlN73K%2f2BcYXln6YUd9YVZ3FLX3OerI69PA%3d%3d](https://www.mouser.com/ProductDetail/Antenova/SR4W030-100?qs=sGAEpiMZZMuBTKBKvsBmlN73K%2f2BcYXln6YUd9YVZ3FLX3OerI69PA%3d%3d) +- [https://www.mouser.com/ProductDetail/Antenova/SRF2W021-100?qs=sGAEpiMZZMuBTKBKvsBmlMeP1Lut7uca61hspfdOxQexT8ZJsKeXqw%3d%3d](https://www.mouser.com/ProductDetail/Antenova/SRF2W021-100?qs=sGAEpiMZZMuBTKBKvsBmlMeP1Lut7uca61hspfdOxQexT8ZJsKeXqw%3d%3d) + +### Power / LiPo Batteries + +- [SparkFun mosfet power control (link)](https://www.sparkfun.com/products/12959) +- [Pimoroni On/Off shim (link)](https://www.adafruit.com/product/3581) +- [SparkFun Charger/Booster (link)](https://learn.sparkfun.com/tutorials/sparkfun-5v1a-lipo-chargerbooster-hookup-guide) +- [Power Meter](https://www.sparkfun.com/products/14331) +- [Pimoroni LiPo shim (link)](https://www.adafruit.com/product/3196) + +### Storage + +- [Ableconn mSATA Hat (link)](https://www.amazon.com/dp/B00WQJ8BH2) +- [Ableconn nvme Hat (link)](https://www.amazon.com/dp/B01LZ0LCTU) + +### Displays + +- [Waveshare 4.2 inch e-ink display (link)](https://www.waveshare.com/product/modules/oleds-lcds/e-paper/4.2inch-e-paper-module.htm?___SID=U) +- [Waveshare 2.9 inch e-ink display (link)](https://www.waveshare.com/product/modules/oleds-lcds/e-paper/2.9inch-e-paper-module.htm?___SID=U) +- [Pimoroni e-ink hat (link)](https://www.adafruit.com/product/3743) + +### Input + +- [Waveshare touch keypad (link)](https://www.waveshare.com/product/RPi-Touch-Keypad.htm) +- [Pimoroni touch button hat (link)](https://www.adafruit.com/product/3472) +- [Pimoroni push button shim (link)](https://www.adafruit.com/product/3582) +- [Adafruit display+button hat (link)](https://www.adafruit.com/product/3531) +- [Adafruit joystick+button hat (link)](https://www.adafruit.com/product/3464) +- [Pimoroni display+button hat (link)](https://www.adafruit.com/product/2694) + +### Cell Data + +- [LTE rPi Shield (link)](http://sixfab.com/product/raspberry-pi-3g-4glte-base-shield-v2/) +- [LTE/4G pciE (link)](http://sixfab.com/product/quectel-ec25-mini-pcle-4glte-module/) +- [3G pciE (link)](http://sixfab.com/product/quectel-uc20-mini-pcle-3g-module/) + +### Misc + +- [SparkFun TTL serial usb adapter (link)](https://www.sparkfun.com/products/14050) +- [Adafruit TTL serial usb adapter (link)](https://www.adafruit.com/product/3309) +- [SparkFun Bluetooth Mate Silver (link)](https://www.sparkfun.com/products/12576) +- [Adafruit GPS (link)](https://www.adafruit.com/product/746) +- [Pimoroni mini hat extender (link)](https://www.adafruit.com/product/3182) +- [Pimoroni hat extender (link)](https://www.adafruit.com/product/3742) + +### 3d Printed Cases + +- Pine64 [(source)](https://www.thingiverse.com/thing:1831345) : ```3d_printer_sources/Pine_A64_Two-Parts_case.zip``` +- Orange Pi Zero [Plus] Base [(source)](https://www.thingiverse.com/thing:2776831) : ```3d_printer_sources/Orange_Pi_Zero_-_Minimal_Mount.zip``` +- Orange Pi Zero [Plus] Case [(source)](https://www.thingiverse.com/thing:1939780) : ```3d_printer_sources/Orange_Pi_Zero_Case.zip``` +- Orange Pi Zero [Plus] + Expansion Board Case [(source)](https://www.thingiverse.com/thing:2353879) : ```3d_printer_sources/Orange_Pi_Zero_+_Expansion_Board_Case.zip``` +- Orange Pi Zero [Plus] + NAS Board Case [(source)](https://www.thingiverse.com/thing:2122451) : ```3d_printer_sources/Orange_Pi_Zero_NAS_Board_Case.zip``` +- Orange Pi PC Case (External Mounts) [(source)](https://www.thingiverse.com/thing:2239240) : ```3d_printer_sources/Orange_PI_PC_Case_with_External_mounts_+_M5_mount.zip``` +- Orange Pi PC NAS Case [(source)](https://www.thingiverse.com/thing:2468854) : ```3d_printer_sources/Orange_PI_PC_NAS_Case.zip``` +- Orange Pi One Case [(source)](https://www.thingiverse.com/thing:1447933) : ```3d_printer_sources/OrangePi_One_Case.zip``` +- Orange Pi One NAS Case [(source)](https://www.thingiverse.com/thing:2790266) : ```3d_printer_sources/Orange_Pi_One_NAS_Case.zip``` +- Orange Pi Zero NAS Case (minimal) [(source)](https://www.thingiverse.com/thing:2740032) : ```3d_printer_sources/Orange_Pi_Zero_NAS_minimal.zip``` +- Orange Pi Zero 2+ H5 Case [(source)](https://www.thingiverse.com/thing:2797865) : ```3d_printer_sources/Orange_Pi_Zero_2+_H5_Case.zip``` +- Orange Pi Zero 2 Case [(source)](https://www.thingiverse.com/thing:2626323) : ```3d_printer_sources/Orange_Pi_Zero_2_Case.zip``` +- Orange Pi Plus 2E Case [(source)](https://www.thingiverse.com/thing:2251219) : ```3d_printer_sources/Orange_Pi_plus_2e_case.zip``` +- Orange Pi Plus 2E Case (alt) [(source)](https://www.thingiverse.com/thing:1916113) : ```3d_printer_sources/Orange_Pi_Plus_2E_case_2.zip``` +- Orange Pi Zero Plus2 H3 Case [(source)](https://www.thingiverse.com/thing:2802598) : ```3d_printer_sources/ diff --git a/hardware/ui.md b/hardware/ui.md new file mode 100644 index 0000000..56f0c8b --- /dev/null +++ b/hardware/ui.md @@ -0,0 +1,19 @@ +Adafruit kit +Connection online/offline +Access point band selection +Modem manager +Poweroff +Reboot +Python + +bluetooth serial console for simple UI to manage things +mimics adafruit kit for overall ui +build underpinnings the same for boh adafruit approach and bt console + +https://www.freedesktop.org/software/ModemManager/api/latest/ + +https://wiki.gnome.org/Projects/NetworkManager/Developers + +https://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/examples/python/dbus + +https://cgit.freedesktop.org/ModemManager/ModemManager/tree/examples/modem-watcher-python/ModemWatcher.py diff --git a/services/acme_sh.md b/services/acme_sh.md new file mode 100644 index 0000000..0c93c8a --- /dev/null +++ b/services/acme_sh.md @@ -0,0 +1,195 @@ +# Let's Encrypt + +Use [acme.sh](https://github.com/Neilpang/acme.sh/) for wholly self-contained Let's Encrypt certificates. This assumes CloudFlare DNS is used for authentication. + +**NOTE:** You probably want to use a DNS provider/API so you don't have to expose a service to the outside world. + +**NOTE:** You may want to use a filesystem on a USB disk instead of /var for the volumes setup in the below Docker command(s) to help reduce writes to the micro sd card. + +## Domains + +### Top Level Domain (TLD) ideas + +- .zone +- .host +- .travel +- .link +- .online +- .net +- .tech +- .club + +### Registrar suggestions + +- [http://namecheap.com/](http://namecheap.com/) +- [https://uniregistry.com/](https://uniregistry.com/) + +## Dependencies + +``` bash +apt update +apt install jq + +``` + +## Prep + +Grab the acme.sh Dockerfile and update it to work with arm (32 or 64). + +``` bash + +mkdir -p /root/docker/acme.sh +cd /root/docker/acme.sh +wget https://raw.githubusercontent.com/Neilpang/acme.sh/master/Dockerfile +sed -i '1s/^/ARG ALPINE=alpine:3.6\n/' Dockerfile +sed -i '/FROM/c\FROM $ALPINE' Dockerfile +mkdir /var/acme.sh +chmod 700 /var/acme.sh + +``` + +## Setup / Run + +Setup a basic update/run script with the adjusted upstream Dockerfile + +``` bash + +cat > /root/docker/acme.sh/acme.sh < To upgrade Borg to a new version later, run the following after activating your virtual environment: + +```pip install -U borgbackup[fuse]``` + +## Initialize Backup Repo + +*Note: assumes you have a ```/tank``` on external disk* + +``` bash + +cd /tank/backup +borg init --encryption none . # No crypto/auth for speed (see docs for more infos) + +``` + +## Backup Script + +Setup a backup script that backs up everything (**note the excludes**) to the initialized repository. + +Run ```/root/borg_backup.sh``` any time you want to take a backup. + +``` bash + +cat > /root/borg_backup.sh < /root/update_caddy.sh < /etc/caddy/Caddyfile < /etc/unbound/local_zone/caddy.conf < /root/docker/nextcloud.sh < /etc/unbound/local_zone/nextcloud.conf < /etc/caddy/services/nextcloud.conf < 'activity',``` to the file. + +#### Add Cronjob + +In the settings change from ```Ajax``` for scheduled jobs to ```Cron``` and run the following commands on your device. + +This will lessen the page loads and keep the cron job constrained to a reasonable duration. + +``` bash + +cat > /etc/systemd/system/nextcloudcron.service < /etc/systemd/system/nextcloudcron.timer < /root/docker/pi-hole.sh < /etc/NetworkManager/dnsmasq-shared.d/pi-hole.conf < /etc/unbound/local_zone/pi-hole.conf < /etc/caddy/services/pi-hole.conf < /root/docker/postgres.sh << EOF +#!/bin/bash + +VERSION="" + +ARCH=\`arch\` + +# Cleanup arch/container image here +if [ \$ARCH == "aarch64" ] +then + echo "64bit arm" + VERSION="arm64v8/postgres:latest" +else + echo "32bit arm" + VERSION="arm32v7/postgres:latest" +fi + +docker pull \$VERSION + +# Cleanup existing container +docker stop postgres +docker rm postgres + +# Re-run/create container with latest image +docker run \\ + --name postgres \\ + --restart unless-stopped \\ + --net docker-private \\ + --ip 172.30.12.12 \\ + -e TZ=UTC \\ + -e DEBUG=1 \\ + -e POSTGRES_PASSWORD=test1234 \\ + -v /var/postgres/data:/var/lib/postgresql/data \\ + \$VERSION +EOF + +chmod a+x /root/docker/postgres.sh + +``` + +## Run Postgres + +Simply execute ```/root/docker/postgres.sh``` to update/run Postgres. diff --git a/services/searx.md b/services/searx.md new file mode 100644 index 0000000..c76540f --- /dev/null +++ b/services/searx.md @@ -0,0 +1,112 @@ +# Searx + +Self hosted metasearch. Prevent profiling by major search engines + +## Inspiration / Further Reading + +- [https://asciimoo.github.io/searx/](https://asciimoo.github.io/searx/) +- [https://github.com/asciimoo/morty](https://github.com/asciimoo/morty) +- [https://asciimoo.github.io/searx/user/own-instance.html](https://asciimoo.github.io/searx/user/own-instance.html) + +## Install / Update / Run Script + +Setup a generic script that'll auto update Searx, build a container and launch it. You should only run this script at first launch and/or when you're looking for updates. + +``` bash + +mkdir -p /var/searx +chown root:root /var/searx +mkdir -p /root/docker/searx +git clone https://github.com/asciimoo/searx.git /root/docker/searx/src + +cat > /root/docker/searx/searx.sh << EOF +#!/bin/bash + +cd /root/docker/searx/src +git checkout Dockerfile +git fetch +LATESTTAG=\`git describe --abbrev=0 --tags\` +git checkout \$LATESTTAG + +ARCH=\`arch\` + +# Cleanup arch/container image here +if [ \$ARCH == "aarch64" ] +then + echo "64bit arm" + sed -i 's_alpine:3.5_arm64v8/alpine:3.5_g' Dockerfile +else + echo "32bit arm" + sed -i 's_alpine:3.5_arm32v6/alpine:3.5_g' Dockerfile +fi + +docker build \\ + --file ./Dockerfile \\ + --tag searx/searx:\$LATESTTAG \\ + . + +# Cleanup existing container +docker stop searx +docker rm searx + +# Re-run/create container with latest image +docker run \\ + --name searx \\ + --restart unless-stopped \\ + --net docker-private \\ + --ip 172.30.8.8 \\ + -e TZ=UTC \\ + -e DEBUG=1 \\ + -e BASE_URL=searx.domain.tld \\ + searx/searx:\$LATESTTAG +EOF + +chmod a+x /root/docker/searx/searx.sh + +``` + +## Run Searx + +Simply execute ```/root/docker/searx/searx.sh``` to update/run Gogs. + +## Serving Via Caddy + +``` bash + +cat > /etc/caddy/services/searx.conf < /etc/unbound/local_zone/searx.conf < /var/syncthing/.config/syncthing/config.xml < + + false + true + false + false + 10 + /tank/syncthing + + +EOF +chown syncthing -R /var/syncthing +chgrp syncthing -R /var/syncthing + +``` + +## Install Syncthing + +Grab the latest release of syncthing, drop it in place, setup system service. + +``` bash + +RELEASE=`curl -s https://api.github.com/repos/syncthing/syncthing/releases/latest | jq -r .tag_name` +ARCH=`arch` +if [ $ARCH == "aarch64" ] +then + ARCH="arm64" +else + ARCH="arm" +fi + +gpg --keyserver keyserver.ubuntu.com --recv-key D26E6ED000654A3E +mkdir -p /tmp/syncthing +cd /tmp/syncthing +curl -sLO https://github.com/syncthing/syncthing/releases/download/${RELEASE}/syncthing-linux-${ARCH}-${RELEASE}.tar.gz +curl -sLO https://github.com/syncthing/syncthing/releases/download/${RELEASE}/sha256sum.txt.asc +gpg --verify sha256sum.txt.asc +grep syncthing-linux-${ARCH} sha256sum.txt.asc | sha256sum +tar -zxf syncthing-linux-${ARCH}-${RELEASE}.tar.gz +mv syncthing-linux-${ARCH}-${RELEASE}/syncthing /usr/bin/syncthing +chmod a+x /usr/bin/syncthing +mv syncthing-linux-${ARCH}-${RELEASE}/etc/linux-systemd/system/syncthing@.service /etc/systemd/system +systemctl daemon-reload +cd ~ +rm -rf /tmp/syncthing + +``` + +## Adjust firewall to allow syncthing on internal network(s) + +``` bash + +firewall-cmd --permanent --zone=internal --add-port 22000/tcp --add-port 21027/udp +# Allow GUI from docker containers (it'll be proxied by the main web proxy container for ssl purposes) +firewall-cmd --permanent --zone=trusted --add-port 22000/tcp --add-port 21027/udp --add-port 8384/tcp +firewall-cmd --reload + +``` + +## Run Syncthing Via systemd Service + +``` bash + +systemctl enable syncthing@syncthing.service +systemctl start syncthing@syncthing.service + +``` + +## Setup Update Script + +Syncthing has an auto update mechanism. Script it so it can be run at any point to get updates. + +``` bash + +cat > /root/update_syncthing.sh < /etc/unbound/local_zone/syncthing.conf < /etc/caddy/services/syncthing.conf < /root/docker/ttrss.sh << EOF +#!/bin/bash + +ARCH=\`arch\` +HUBIMAGE="" + +# Cleanup arch/container image here +if [ \$ARCH == "aarch64" ] +then + echo "64bit arm" + HUBIMAGE="lsioarmhf/tt-rss-aarch64:latest" +else + echo "32bit arm" + HUBIMAGE="lsioarmhf/tt-rss:latest" +fi + +# Cleanup existing container +docker stop ttrss +docker rm ttrss + +# Re-run/create container with latest image +docker run \\ + --name ttrss \\ + --restart unless-stopped \\ + --net docker-private \\ + --ip 172.30.13.13 \\ + -e TZ=UTC \\ + -e DEBUG=1 \\ + -v /var/ttrss:/config \\ + \$HUBIMAGE +EOF + +chmod a+x /root/docker/ttrss.sh + +``` + +## Run TT-RSS + +Simply execute ```/root/docker/ttrss.sh``` to update/run TT-RSS. + +## Serving Via Caddy + +``` bash + +cat > /etc/caddy/services/ttrss.conf < /etc/unbound/local_zone/ttrss.conf < /root/docker/wallabag/wallabag.sh < /etc/caddy/services/wallabag.conf < /etc/unbound/local_zone/wallabag.conf <