Skip to main content

Home Datacenter Overview

I thought it would be fun to do a write-up of my home datacenter, so here you go.

Compute

Datacenter hardware is expensive, so my compute is actually consumer products. I built 3 computers with the following specs:

  • CPU: Ryzen 5 7600X 4.7 GHz 6-core / 12-thread
  • RAM: Corsair Vengeance 64GB DDR5-6000 CL30
  • PSU: Corsair SF600 600W 80+ Platinum
  • OS Data: Kingston DC1000B 240 GB NCME SSD
  • Price: ~$950 each (not including additional storage)

These computers make up a 3-node Proxmox cluster that I use to run Linux Containers and Virtual Machines. Proxmox allows for "high availability" meaning if one of my Proxmox nodes goes offline, the VMs and Containers on that node will automatically start up on another node.

Future Plans - Compute

I don't have any plans to upgrade my CPU, RAM, or PSU. They are meeting my current needs and I don't expect to outgrow them anytime soon.

Storage

When designing my ideal storage system, I had 2 main requirements:

  1. Data resiliency in the event of drive failure
  2. Data availability in the event of node failure/shutdown

Previously, I had 3 HDDs with all of my data. There were no copies or backups of my data, so if any of the drives failed then my data was gone. If my one storage node went offline, my data was unavailable to me until I got it back online. It wasn't ideal.

My solution was to build a 3-node Ceph storage cluster using the same custom-built computers running my Proxmox cluster. I have Ceph designed to replicate my data to all 3 storage nodes, so any 2 of them can go offline without losing data availability. Additionally, my data can survive up to 3 simultaneous drive failures without risk of loss.

Each storage node has the following drives:

  • 3x Western Digital Red Pro 14TB 7200 RPM HDDs
  • 1x Kingston DC500M 960GB SSD
  • Price: ~$875 each

Future Plans - Storage

I would like to add more SSD storage to use as a caching-tier for the HDD storage for faster write speeds, but Ceph basically requires datacenter-grade SSDs which are expensive.

Other than that, my storage is about 70% utilized and I've had to delete things to get down to that percentage. I'd like to add more HDD storage, but my cases are at capacity. I'd have to build additional storage nodes in order to add more storage.

Network

I have what I'd consider to be an enterprise-grade network.

  • 2x virtualized OPNSense routers/firewalls in primary/failover mode
  • MikroTik CRS328-24P-4S+RM network switch
  • 2x TP-Link Omada EAP655-Wall Wireless Access Points

Since my ISP package is only 1G up/down and most of my devices only support 1G wired network speeds, the OPNSense firewalls are only wired with 1G ports. However, the 3 compute/storage nodes are wired together using 10G SFP+ ports. This is recommended for Ceph storage clusters, especially when utilizing SSD / NVMe storage drives.

The 1G ethernet ports on the switch also support Power-over-Ethernet, which I use to power my security cameras and my wireless access points. I've got one access point downstairs and the other upstairs.

Future Plans - Network

There are 2 upgrades I'd like to make to my network. For starters, I'd like to buy a MikroTik CRS309-1G-8S+IN switch to use as a dedicated backbone for my compute/storage nodes. This is recommended for Ceph storage clusters so that the data replication between servers can happen on a separate channel from the user requests. It's $270 and hasn't been a high enough priority yet.

The other upgrade I'd like to make is upgrading my ISP package from residential to business so I can have a dedicated /3 subnet or maybe even a 10G up/down fiber connection instead. I'd have to request a quote, so I'm not sure how expensive it would be.

Services

"What do you do with all of this?" is one of the most common questions I get when I talk about my setup. The truth is, I did this all to learn how to do it. I've since convinced my company to implement similar technologies at work, so there's definitely been benefits to the professional development. I do use and host services at home though.

  • Omada Software Controller (Wireless Access Point Controller)
  • Frigate (Network Video Recorder for Security Cameras)
  • Home Assistant (Home Automation)
  • Vaultwarden (Password Manager) - Implemented at work
  • Servarr Stack (Media Management)
  • Plex (Home Media Server)
  • Mealie (Recipe Manager)
  • Zabbix (System Monitoring) - Implemented at work
  • Gitea (Code Tracking) - May implement at work
  • Personal Website (This Website)
  • Various Game Servers (Minecraft, V Rising, etc.)

Future Plans - Services

I eventually plan on implementing and using a photo/video management service. I'd also like to self-host my own home security / alarm system. I need to buy a smart door lock and a security panel compatible with home automation. I haven't picked them out yet, so I'm not sure how expensive they'll be.

Back