Raspberry pis are an easy intro to actually using computers (instead of using something like windows).
Raspbian is great (based on Debian) and there is a HUGE community for it.
So yeh, it's a great started for $25, as long as you have a PSU and SD Card. And an hdmi cable + monitor + keyboard at your disposal (and a mouse if you are installing a desktop environment (IE something like windows, whereas headless is a full screen CLI).
And don't get your hopes up for a windows replacement.
But.... Why not run a Virtual Machine? If you have a windows machine, run VirtualBox, create a VM and install Debian on it?
That's free. You can tinker and play.
And the only thing you are missing from an actual raspberry pi is that it isn't a standalone device (IE your desktop has to be on for it to be running), and it doesn't have GPIO (ie hardware pins. And if this is your goal, there are other ways).
If you really really want a computer that is on all the time running Linux (Debian, a derivative (like raspbian) or some other distro) - aka a server - then there are plenty of other options where the only drawback is lack of GPIO (which, in my experience, is rarely a drawback).
And that is literally any computer you can get your hands on. Because the raspberry pi trades A LOT for its form factor, the ethernet speed is limited, the bus speed is limited (impacting USB and ethernet (and ram?)), the SD card is slower and will fail faster than any HDD/SSD. The benefit is the GPIO, the very low power draw, and the form factor - rarely actually a benefit.
I'd say, play around with some virtual box VMs. See what you want, other than Fear Of Missing Out (things like PiHole? They run on Debian, or even in a docker container). Then see if you actually want a home server, and what you want to run on it.
It's likely you won't want a raspberry pi, but a $150 mini pc that can actually do what you want.
Oh, and on the "fail often" thing...
Get a basic/old/free pc/laptop and install Proxmox on it.
Loads of tutorials out there, but the basic installer will get you to a "I'm learning" stage.
Create a VM, install Debian, play around.
Then: create a new VM, install Debian, create a snapshot, play around until it does what you want, restore the snapshot, do the steps that got you from vanilla to what you want. Create snapshots along the way as checkpoints. Snapshot, tinker, restore snapshot, advance.
Proxmox is amazing for learning VMs and server things