Selfhosted

63318 readers

434 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Feasibility & sanity check: Scaling a P330 Tiny to a 3-node Proxmox cluster (lemmus.org)

submitted 1 month ago by buildmylab@lemmus.org to c/selfhosted@lemmy.world

14 comments fedilink hide all child comments

I’m currently running Proxmox on a single Lenovo ThinkStation P330 Tiny. As my VMs workloads have grown, the 64GB of RAM is consistently sitting near capacity.

To spread the memory load and get proper HA, seamless VMs migration, and easier node maintenance, I’m considering adding two more matching units to build a 3-node cluster.

Planned Per-Node Specs:

. Chassis: Lenovo P330 Tiny . CPU: Intel Core i9-9900T ES . RAM: 64GB DDR4-3200 (running at 2666MHz platform limit) . Boot/Local Storage: 1TB NVMe

Networking: Upgraded to 2.5GbE

Shared Storage:

Synology NAS via iSCSI (SSD cache enabled) over 2.5GbE

Does this setup still offer good value for money, or are there better alternatives in this form factor?

I’d especially appreciate feedback on:

9900T ES stability: Long-term reliability and virtualization quirks under Proxmox.
2.5GbE bandwidth: Is it sufficient for 3 nodes sharing iSCSI, especially during backups or migrations?
Synology iSCSI + SSD cache: Practical experience with this as primary VM shared storage across multiple P330 tiny nodes.
Any other bottlenecks, hidden traps, or limitations I might be overlooking?

you are viewing a single comment's thread
view the rest of the comments

[–] ryan_@piefed.social 1 points 1 month ago* (last edited 1 month ago) (1 children)

I have a similar setup, 3 mini pcs (2 dell, 1 hp) with a second 2.5Gb interface on each and a synology nas.

Initially I used iscsi for storage but I had a couple of annoyances that I encountered:

At that time, taking snapshots on vms using iscsi storage wasn’t possible.
My Synology NAS doesn’t have a way to natively install a 2.5Gb interface, so I used a USB 2.5Gb NIC. Occasionally (like one or twice per year, from my recollection) the NIC would stop working and my VMs would lose their storage. Easily fixed by rebooting the Synology and VMs, but annoying none the less.

Because of that I ended up installing an SSD in each node and creating a CEPH storage cluster. No more issues with VMs losing storage and it gave me the ability to create snapshots.

My current environment is 17 LXCs and 4 VMs living in a ceph cluster that runs on each proxmox node. Using PBS, I have backup jobs scheduled across 4 days each week, 2 of the days backup to my nas and the other 2 backup to Backblaze.

I’ve never encountered any noticeable bottlenecks with my current environment so I think you’ll be just fine

[–] buildmylab@lemmus.org 1 points 1 month ago

Thanks, this is really helpful feedback! I actually never considered USB NICs on the Synology because of the extra power draw and stability concerns, so sticking with native dual 1GbE (via LACP or MPIO) into a 2.5G switch is definitely my plan.