On occasion I find myself needing to send a file at least a few gigabytes in size to a friend across our slow ISPs but haven’t found a satisfying solution. I usually end up creating a private torrent with the announce address of my own IP. Even though it’s slow - it basically never reaches my max upload speed for some reason, it is at least resilient if there are ever any network glitches.
Does anyone else face this same challenge?
EDIT: Thank you for the awesome suggestions! I have some homework to do on these
Syncthing
Er, wait, are you using Syncthing for its intended purpose of syncing files across devices on your local network? And then exposing that infrastructure to the internet? Or are you isolating Syncthing instances?
Syncthing is not limited to local network. It’s hole punching is one of the major features
Yep, I’ve got a buddy in another country that I needed to share a group of files with, it was several gigs and we were both editing things.
We setup a syncthing connection and once we were synced it just worked. I also use it on my LAN to sync personal files, but to share with him we both just set up a folder and I just shared that one folder with him while the rest of my shares stayed private on the LAN.
Syncthing is amazing.
The fact that Syncthing seems to solve CGNAT on its own has me wondering why there are not more solutions for the server/home side.
Why does Wireguard or any other VPN not work like Tailscale or Zerotier?
Why don’t torrent clients can’t work with IPv6 to seed more?
Why doesn’t Plex adopt a similar mechanic like Syncthing to expose the media over the Internet instead of being a prisoner of CGNAT?
I know I am just throwing different options with my personal frustrations lol, but I hope you get what I am trying to mean, Plex, torrent and home VPN users shouldn’t become masters at networking, especially when the documentation for the tools IS NOT ENOUGH.
Why does Wireguard or any other VPN not work like Tailscale or Zerotier?
tailscale and zerotier are wireguard, but with a public server that helps with NAT. Syncthing uses a public server for that too.
wireguard was specifically made to be as simple and minimalistic as possible.
Why don’t torrent clients can’t work with IPv6 to seed more?
is there such a problem? honest question. But I think that might be a different issue
Why doesn’t Plex adopt a similar mechanic like Syncthing to expose the media over the Internet instead of being a prisoner of CGNAT?
maybe they just don’t see working on it profitable enough
> is there such a problem? honest question. But I think that might be a different issue
Yes, that is a problem. We’re still in a world where you need to manually enable port forwarding in order to get better seeding for bittorrent clients, and if you have CGNAT you’re SOL (short of using a VPN or something to bounce through an external host).
It’s likely because torrent software is older (& in crappier languages), and came about before CGNAT was a thing.
> is there such a problem? honest question. But I think that might be a different issue
Yes, that is a problem. We’re still in a world where you need to manually enable port forwarding in order to get better seeding for bittorrent clients, and if you have CGNAT you’re SOL (short of using a VPN or something to bounce through an external host).
I don’t understand, sorry. they were saying that something doesn’t work as expected IPv6. but CGNAT is not used for IPv6, is it? and you don’t really forward ports either, maybe you allow them through in the routercs firewall but notnsure because I don’t have v6
We’re still in a world where you need to manually enable port forwarding
well, you don’t need to, often you can also enable the upnp function in the router so that any software can open all the ports it wants, which is a terrible idea security-wise
Syncthing is not just for LAN use. Even their homepage mentions transmitting data over the internet
I’ve been using it to sync devices over the internet for years. It’s also how people use it to sync from say their desktop to their phones, remote server, etc.
If you watch your network firewall Syncthing does reach out to servers on the internet to help it find other devices so e.g. if you enter the other device’s ID (example ABCDEFG-ABCDEFG-ABCDEFG-ABCDEFG-ABCDEFG-ABCDEFG-ABCDEFG-ABCDEFG) it can reach out over the internet to find that specific ID. I think Syncthing uses a sort of DHT resolver to find other devices, I know on my firewall I had to whitelist Syncthing’s servers to make it work.
I was going to try to link you some references but their forums seem to have connection issues at the moment, you may want to search around later if you’re interested how Syncthing works over the internet.
You’re probably thinking of PairDrop (which also allows extranet).
The user can choose. Please note you first much accept another client by its fingerprint.
Syncthing has public releays enabling it to work (dunno if one or none need to be public) without both parties being exposed.
It’s very much a WAN solution too. I use it to push my files to a Pi Zero W that’s 200 miles from my house. I use it as an off site store of my files. The Pi is connected as an untrusted device in Syncthing so that all files sit encrypted at rest.
You can use syncthing to transfer files across the internet? How? I thought it was only for local networks
It just works, there’s no “how”. Take one of the devices outside, connect to the internet, done.
My largest file transfer I have done via USB disk. You simply don’t transfer multiple terabytes over the net.
Super easy. Spin up an OpenVPN server, forwarding the right ports to your server. Now spin up an Apache server with the folder your file’s in as server root. Send the client config for your VPN to your friend, along with the local address of your HTTP server. Now they can install the OpenVPN client on their PC and download the file from your HTTP server. Once you’re done, tear down all your servers, and don’t forget to unforward the ports. Couldn’t be easier.
/s
Okay can you explain why thats a sarcastic answer? Is one of those first three steps way harder than I think it is?
Cause that’s not simple or easy at all. It takes a fair bit of knowledge to set up all of these things.
RFC 2549: https://www.rfc-editor.org/rfc/rfc2549
This problem was solved years ago: https://spectrum.ieee.org/pigeonbased-feathernet-still-wingsdown-fastest-way-of-transferring-lots-of-data
You could try wormhole. It makes a direct connection.
Or croc which is very similar. I think it also allows to resume file transfers.
Reminds me I had been needing to find something for this too. Looks like I had thought about using Croc.
Wormhole.app, can’t recall if they have a limit atm
I’d go for syncthing over nextcloud for your specific usecase. Nextcloud isn’t good for unreliable connections and they’re sticking with the annoying decision of not supporting server to server synchronization.
I’d have to have friends across the internet that wanted files first…
Not sure if this works for you but I didnt see it mentioned. I use plex for my media server, so I would just put whatever it is on there and then someone else can log in remotely and download it through the app on their mobile, and I think also via the website too.
I know this works if the person is downloading from android but haven’t tested otherwise.
That should work for media files at least, but I believe they’ll also need Plex pass to be able to download anything.
Should be able to do that with Jellyfin, no Plex/Plex Pass needed. That said I suspect your current method with creating a torrent to share is much more resilient when dealing with choppy internet connections. With Jellyfin/Plex it’s more of a direct download situation, not sure if either can resume broken downloads.
i like zipline but i use it for smaller files where download resumability is not a key factor
I’ve used:
But for slower connections bittorrent is the best option by far because it doesn’t care about interruptions, and verifies the data as it goes. Just gotta make sure you’re port forwarding the client.
