Now it would be interesting to setup a raspberry pi with harddrives plugged in the USB 3 portsπ‘
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (donβt cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
π€―
Elderly raspberry pi B [β]
Large portable drive gathering dust [β]
Guess I'm setting up a locally hosted file server in the near future.
Everyone loves CP. Tell all your friends about CP.
I have a question, and I want to emphasise thar this is not criticism but a request for dive into technicalities.
In the video you mentioned copyparty has an one-way sync tool. Is there a good reason why it's not two-way, or is this just something you weren't motivated to do?
No worries, good question :>
The problem with bidirectional filesync is that it's an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There's an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone's vacation photos, or suddenly downgrade someone's keepass database to an older version... And stuff like syncing multiple devices to the same server makes it balloon further.
I've started becoming more confident in copyparty's filesystem-index database, but it's still just a hint/guideline, with the filesystem being the only source of truth -- it's still not something I'd trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I'd like to leave this in their capable hands.
Thank you for your answer! Do you think copyparty would work together with Syncthing on the same backing directory, or would they compete for changes etc? Copyparty in this scenario would be for sharing content with friends and occasional remote upload
that should be totally fine, I think a lot of people are doing that :>
Look cool! I think you should consider putting a screenshot of the UI somewhere near the top of the README
Oh my gawd what a README!! I'm on my phone and I was trying to scroll back to the top of it from the bottom and I just kept on scrolling... Holy shit I'm going to put this on my kanban board give it proper attention
At this rate might be faster to read code than a read me. Or convert to a wiki style if this much details are really needed.
Haven't looked at the project yet, but that's just the greatest name for a fileserver...
Can you point me to the WebDAV code? Iβm interested to see your implementation. There are some parts of the spec that are ambiguous, and I like to see how those are implemented in different servers.
sure! my implementation is really basic, just the stuff that's needed to make the clients i've tested happy, so there's probably still clients that won't be able to connect (And i'll fix those as soon as I hear about them!)
httpcli.py is the http methods handler, and the webdav-specific handlers are all next to eachother, propfind // proppatch // lock // unlock // mkcol // and there's also put for the uploads, but that's not entirely webdav-specific, just webdav-aware.
Very sleek project. The language switcher bit was brilliant hahaha. Seriously, good job.
The fact you mention security features, without ever saying it's 'super secure' tells me you know a lot about what you're doing. I'm so sick of apps like this that start with "most secure app on the net" but you know they're delusional. Thank you, going to check this out.
You made this on your phone on the bus ride to and from work.
I cleaned the cat box yesterday and considered that an accomplishment.
Fuck.
Looks fantastic, I'll actually be trying this. Love how it doesn't lock my files into some obscure format like seafiles.
Hey fellow scener, cool project!
Just a few thoughts/questions:
- BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.
- Why have it be one huge python source file? This is a serious code smell imo, and something you really should avoid doing as this can be a major maintenance burden.
Just a remark from someone who runs ZFS since the beginning. Many people don't like the deduplication feature because of its memory footprint.
It's also nice to have this feature without relying on a certain filesystem.
BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.
yeah that's a good point, I'll add an option to take advantage of this if you know you're running on a filesystem where that works as intended.
Why have it be one huge python source file?
oh don't worry, it's all separate files during development -- there's a build-stage which bundles everything up into a single file for distribution. But thanks for the concern :D
What do you use to bundle into one file?
copyparty-sfx.py
is a custom packer (see this reply) created by make-sfx.sh, and copyparty.pyz
is a standard zipapp, created by make-pyz.sh. The zipapp has more disadvantages than the sfx.py
, so that's the default/recommended build.
Ah, so you have compiled it into one file? Didn't know that was possible for python, what tool do you use for this?
sooo this is one of the things that started with someone saying "wouldn't it be funny if..."
if you open copyparty-sfx.py in a text editor, you'll see how -- but please make sure to use an editor which is able to handle about 600 KiB of comments which contain invalid utf8 / binary garbage π
I ended up rolling my own packer since I wanted optimal encoding efficiency, and everything I could find would do stuff like base85 or ucs2 tricks, but it turns out python is perfectly happy with binary garbage in comments if you declare that the file is latin-1
so it realizes all hope is lost :D
the only drawback of the sfx.py is that it needs to extract to $TEMP before running, so that's the slight advantage of the zipapp (the .pyz alternative), but that suffers from some performance reduction in return, and is more hermetic (doesn't let you swap out the bundled dependencies with fresh versions as easily if necessary)
you even mention ladybird as browser, nice π
it's such an impressive project! Amazing what they've accomplished in so little time, and so important too -- we need as many options as we can get.
I agree but it's still in an early development state. Not really usable for everyday work let alone most people never heard about it π But yeah still cool to mention it under "modern" browsers. I wish them good luck with the first alpha next year. I hope it'll be successful.
This is really impressive
Your readme looks super in depth, thanks for that! I haven't watched the video yet but will later.
I didn't see it mentioned from a quick glance, but is either sftp or ftps supported?
SFTP is not currently on the roadmap, but it's not entirely implausible.
FTPS is supported, but it requires an optional dependency to be installed (pyopenssl), so it's not available in the Windows EXE. And I just realized that the dependency is currently not present inside the docker images either, so I'll get that fixed right away.
Screenshots of the ui at the top of the readme would be nice
This looks great but I really wish it had SFTP.
thanks for the vote, I hear ya :>
Alright, you bought me with this
Amazing presentation and nice that you have a demo!
Great job on something like this! I'll probably give it a whirl soon, I like Nextcloud but find it clunky sometimes because it's often a bit more than I need. Maybe breaking it up into Immich + this would help! Thank you for sharing your project!
One thing to note, your comparison against Nextcloud has a partially-incorrect point regarding file upload max size. The client does upload chunking, so is unaffected by the Cloudflare issue as well, but I believe the web client is still affected, just not the apps. https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html#adjust-chunk-size-on-nextcloud-side
I suspect a few others may be as well, but I'm only familiar with the Nextcloud one because that's what I've been running, and discovered in making sure I could still upload video files recorded while out and about.
Also love that it looks like a simpler install!
Thanks for the correction; confirmed that Nextcloud now does chunked uploading (with the web-client too). Fixed :>
Good luck, and let me know if you hit any issues o/
Clearly a labour of love π
Maybe support for some music streaming apps (subsonic?) would be cool?
This is very impressive and I'm highly likely to give it a whirl. My question is, though: would it be something that my very non-tech savvy wife could use?
Eg. I'm thinking setup the app on her phone with a default location and when she asks me for a file I can just tell her that I've "put it in the app", and she'll be able to easily retrieve it. Also same thing but vice versa, though the video seems to cover that via the Android share menu...
Again, super impressive. Good job!
I have a hunch that the true answer, to be honest, is "no" -- at least with the current UI as it is. I've come to terms with not being the best at making intuitive user interfaces, so I went all-in on making it poweruser-friendly and efficiency instead.
Yeah, there's the android app for sending files to the server, and it'll always send files to the same folder, so that part should be pretty solid. But actually grabbing files from the server, perhaps not so much. Not sure I'd risk it, but I'll leave the decision to you hehe
I'm not aware of any user-friendly android/iOS apps for connecting to a webdav / ftps / sftp server, but if those exist, then that would probably have been a good option!
Oh my god, this seems really good and closer to what I want than anything yet. Been looking for something to replace Nextcloud and found nothing good so I might take a look at this.
Any way to run the server as a docker container?
Yep -- https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker
Hopefully that description makes sense (let me know if it doesn't)
You can run absolutely anything as a docker container that you have the binary (and other files if needed), or you can go fancy and compile from source in docker.
Just create a dockerfile.
From (some base image you want to use like Ubuntu or Alpine)
Copy necessary files
Run the binary
You can run it straight from command line, put it in a docker compose file, or even tag it and upload it to a repository (and then reference that in your docker compose)
OMG! I've been looking for something like this for quite some time!
I will try this as soon as I have time. Thank you!