Hello,
I’m looking for guidance in setting my home server. I have a HP Proliant Gen 8 Microserver G1610T box upgraded with 12GB RAM.
I was using it and plan to continue using it with some critical applications AND as a homelab to test an play with new OSes, technologies and solutions.
TL;DR
If this post is too long for you, just please look on the diagram and the questions below it.
Background
In short, my plan is to isolate my production environment from testing ground, so I won’t mess up with services that should be up and running most of the time. Also, I am occasionally away from the physical box so if I screw up network configuration (which I’ve done…) or my host won’t boot up I’m f.u.b.a.r.. For that reason, I decided to have one VM for all critical applications and others for testing, according to my needs.
What I want
- Separate testing environments from services I don’t want to break
- Have remote access to administer machine, install / reinstall OSes
- Use snapshots to have point in time backups that I can revert to if something goes wrong
My plan
To achieve further fail-proof isolation and preserve from config contamination, I want to put those critical application in containers, preferably LXD ones because I then can provision them with ansible without much tweaking (the same way I’d done without any layer of vm / container).
Because my server is in my home behind my home router in order to access it from the internet I have to forward ports to it, so I my solution is as follows
- Port 80 and 443 for web applications are pointed to a container with nginx serving as reverse proxy (pointing cloud.mydomain.com to container 2, git.mydomain.com to container 3 etc.).
- All other applications are separated in LXD containers according to their purpose
- Other services (like XMPP) are forwarded (by router directly to specific container
- Bare-metal host OS is Ubuntu 18.04 installed on LVM to achieve easy rollback and backup with snapshots. Livepatch and unattended upgrades are setup. Host OS have minimal set of applications installed and all ports except SSH opened. For visualization KVM is used.
- On VM PROD another Ubuntu 18.04 is installed with latest snap LXD and several per-service containers.
- Both HOST and VM prod have bridge setup so every new VM and container can have its own IP on my LAN so further configuration will be easier.
- Each single machine (VM and container) is provisioned by Ansible with separate playbooks.
- For snapshots and backups ZFS is used for containers and LVM for VM PROD and HOST
- Storage consists of 1 SSD for HOST OS with setup and 4x3TB WD RED drives. I have not decided how to set them.
Below I attach a diagram of what I plan to setup and would like to ear some feedback about it - what are possible pitfalls and drawbacks and what should I do differently.
.-----------------.
| HOST |
| (on bare metal) |
| __________ |
| [_|||||||_°] |
| [_|||||||_°] |
.------------------------------------->| [_|||||||_°] |-----------------------------------------------------.
| |-----------------| |
| | Ubuntu 18.04 | |
| | on LVM | |
| | +br0 bridge | |
.---------------------. '-----------------' v
| VM PROD | .------------------------.
| production services | | other VMs |
|---------------------| |------------------------|
| Ubuntu 18.04 on LVM |<----------------------------------------------------------------------------------. | for testing |
| with LXD containers | | | | | | and learning purposes |
| on ZFS | | | | | | (CentOS, pfsense etc) |
| +br0 bridge | | | | | '------------------------'
'---------------------' | | | |
^ | | | |
| | | | |
| | | | |
| .----------------------. .--------------------------. .-----------------------. .-----------------.
| | LXC container 2 | | LXC container 3 | | LXC container 4 | | LXC container 5 |
| |----------------------| |--------------------------| |-----------------------| |-----------------|
| | nextcloud | | web applications: | | XMPP server (prosody) | | SAMBA |
| | (cloud.mydomain.com) | | gitea, wallabag etc | | (xmpp.mydomain.com) | | (LAN only) |
| '----------------------' | (git.mydomain.com, etc.) | '-----------------------' '-----------------'
| ^ '--------------------------' ^ ^
| * ^ * |
.---------------------. * * * | Home PC,
| LXC container 1 | * * * L| laptop etc
|---------------------|********************************************************** A|----- __ _
| nginx reverse proxy | N| [__]|=|
'---------------------' | /::/|_|
^ |
* | .-,( ),-.
* access from INTERNET Home router .-( )-.
************************************************************************************** __________ <*****( internet )
[_...__...°] '-( ).-'
'-.( ).-'
QUESTIONS
- How to orchestrate / administer it? Since every container / VM is a full separate OS, how do I batch upgrade them? With Ansible?
- HP Proliant has two NICs - how can I isolate home LAN (samba) with internet?
- Should I put samba on HOST rather than in LXD for performance reasons?
- How do I manage storage? To expose folders/ disk to containers I have to share it with VM PROD and then with LXD - is there a big penalty in I/O for it? Can I go with ZFS (since HOST is Ubuntu 18.04 which supports it without DKMS)
- Is there a big performance fall in setting ZFS storage for LXD containers on LVM?
Any additional comments and shared experience would be much appreciated.
Makes me feel I should add some more hardware to my home lab.