Infrastructure on Obsessive Computer Documentation

Apache

Mon, 01 Jan 0001 00:00:00 +0000

Everybody wants to rule the world, but Apache actually does

Apache is the granddaddy of open source projects - and probably one of the simplest. It’s a fileserver, it takes files from your hard disk and gives them to people who ask for them.

Since you might not want to do that for every file, it has a set of rules for which files it can serve, from where and who to

Bitwarden

Mon, 01 Jan 0001 00:00:00 +0000

My bitwarden setup

Docker

Mon, 01 Jan 0001 00:00:00 +0000

Solving your deployment problems

Docker is a program that runs on your machine which allows you to run other, virtual machines. These virtual machines (“containers” in docker-speak) are isolated from your machine (“host” in docker-speak).

A container has an operating system (almost always Linux) and comes with it’s own private filesystem. They’re normally configured to access the internet through the host network.

Technically, Docker is just one program of many that can run OCI-compliant (Open Container Initiative) containers. For example, Kubernetes can also run containers. Docker is a lot simpler than Kubernetes, and as such has a significantly shallower learning curve. Plus, it’s extremely popular.

Filebrowser

Mon, 01 Jan 0001 00:00:00 +0000

Nobody likes FTP

In the dim and distant past of the internet, we had different protocols for different things - and FTP was for transferring files.

Nowadays everyone uses a web browser, and so of course all those FTP applications (with universally terrible UI’s) have been replaced with web page apps using HTTP(s)

Filebrowser is one of those web page apps. It has a lovely UI. You install it somewhere, and it gives access to a portion of your hard disk to anyone with a web browser and a password.

Git

Mon, 01 Jan 0001 00:00:00 +0000

Wherein it becomes apparent that upsetting one of the brightest developers in the world is a very bad idea.

A brief history lesson

In the bad old days, source control was client-server. You, the client, talked to the single central server to check out your files, and once you’d modified them you told the server about your changes.

In the mean time, nobody else could change those files on the server - they were locked out.

Gitea

Mon, 01 Jan 0001 00:00:00 +0000

Because there’s no such thing as a free lunch

There are a number of organisations (bitbucket, gitlab, github, etc) who will run a git server for you. Most of them have a free or freemium tier, but these tend to be free in the sense of “the first one’s free”.

You can instead self host your own git server: https://about.gitea.com/

Here’s my docker-compose.yaml, as per the official instructions

If you want to keep any data then the volumes must to be edited to match what you’ve got available on your docker host machine

Immich

Mon, 01 Jan 0001 00:00:00 +0000

Accessing my photos from anywhere

There’s a free service from Google, called Google photos. This allows you to put your photos onto a private place on the web, and share links to them - but it’s limited to 15Gb.

I’ve got way more than 15Gb of photos, and I’m not paying Google for something I can host myself.

Immich is an open source, docker-deployed image hosting platform which is sufficiently efficient to run on a Raspberry Pi 4. I’ve got plugged into an external HDD of 4Tb

Infrastructure

Mon, 01 Jan 0001 00:00:00 +0000

It’s turtles all the way down. This is how they’re stacked

`````

Servers

homeassistant

Always on. SSH provided by HA plugin - firewalled? Web access

Static IP: 192.168.0.200

mDNS: homeassistant.local

Home Assistant

Details

immich

Always on. Public server. SSH running

Cockpit

Special requirements

External disk (/dev/sda2) needs manual intervention after a power outage

Static IP: 192.168.0.201

mDNS: immich.local

Immich

Integrated Development Environments

Mon, 01 Jan 0001 00:00:00 +0000

Capturing developer eyeballs since forever

Eclipse

IntelliJ IDEA

Visual Studio Code

Remote-SSH

What if the files aren’t on your own local hard disk, but on a different machine? You could copy them locally, or you could use SSH to do this invisibly for you.

This is what the Remote-SSH plugin does - it acts as a filesystem, so the remote files look like they’re local.

Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Tetris for computers

Let’s assume you’ve got a bunch of applications running in docker containers, and a number of computers (aka nodes) to run them on.

How are you going to decide which docker container to put on what node?

This implies you have a lot of containers and a lot of nodes - otherwise why bother? This pattern is also known as “cattle not pets” - you need a lot of livestock before you should bother. This has led to a meme of “I deployed my blog on Kubernetes”, where there’s this whole infrastructure for a single small item.

OCI

Mon, 01 Jan 0001 00:00:00 +0000

Your first evil is free

Oracle have their own cloud offering, similar to AWS and Microsoft. As they’re not very popular, they offer some services for free, forever.

While these instances are tiny, they’re quite useful to host small things, and as a public portal to my home-based infrastructure, via Wireguard

Account

Account name = dibblengrub

Username = dibblengrub@gmail.com

You will need the password from Bitwarden and the 2fa token from Authenticator

Photobooth

Mon, 01 Jan 0001 00:00:00 +0000

Photos in parties

Who wants to spend their time taking photos in a party when you could be doing better things? Why not have a ruggedised, self serve photo booth so everyone can take their own shots.

The photobooth is an rPi v3 with a high quality camera board, a manual-focus 6mm lens and a single ruggedised button configured as a usb keyboard. On boot it runs some hacky python code

Spark

Mon, 01 Jan 0001 00:00:00 +0000

No such thing as “too big”

Originally developed by UC Berkley (who have quite a heritage) it’s now the worlds’ favourite way of processing data that’s too big to fit on one machine.

Docker setup

Architecture

First, you express what you want to happen in SQL, Python, Scala, R or Java. These are all equivalent, but Spark itself is written in Scala.

Execution plans

From your code Spark generates a series of execution plans.

VNC

Mon, 01 Jan 0001 00:00:00 +0000

Accessing servers UI from anywhere

I prefer to use SSH to access a machine. However, sometimes you need to use a UI

VNC is the client (running on windows) that allows you access a remote desktop from a linux machine.

You’ll need the desktop running on the server - if it’s not already booted then use sudo startx

Captive portal

Set up the rpi as an access point

Wireguard

Mon, 01 Jan 0001 00:00:00 +0000

Everyone’s a network engineer now

Wireguard is a lovely bit of open source code for Linux machines, which creates a VPN (virtual private network) with very little effort

sudo apt install wireguard

Background

You’re on machine A, and you want to get to machine B. Wireguard will set up a tunnel between A and B, so that they appear to be next to each other, even though there may be machines inbetween that are owned by hostile actors.