Cross platform Cloud Storage Options

Usually, if someone blocks access to something, you know it has to be good. :smile:

4 Likes

So far so good with Megasync - I’m actually syncing my actual Dropbox folder to a Ubuntu laptop that doesn’t have the Dropbox client installed and it works…

Meanwhile :

1 Like

Necro-posting - rather than start a new thread…

I’m back cloud-service-hopping again :smiley:

I’m still using ResilioSync - love it - will continue to use it. I’m looking for something to use in tandem with ResilioSync. RSL sync is a bit clunky on Android, and not quite as clunky, but still a bit, on iOS (iPadOS)…

Note - despite above earlier post from years back - I stopped using MegaSync - mainly because I’m LOATHE to use it in “Corporate” environments - due to Corporate tossers and Management wankers frowning on stuff that makes the news headlines where they send in SWAT teams to arrest the company founder (Kim Dotcom in New Zealand). I know some companies will block megasync access. And you can say what you like, I know megasync is still widely used to “share” copyrighted or DRM stuff - and - dodgy stuff like pr0n and worse…

Potential Candidates :

  • iCloud (my daughter now works for Apple and we have family sharing and we get 2 TB of cloud storage)
  • Google Drive
  • Dropbox
  • pCloud

What I want to be able to do :

  • Seamlessly access files across Linux (x86 - AND armhf / arm64), MacOS, iOS / iPadOS, Android, Windows (just the one Windows device, at work)
  • Easily and adhoc share files with family, e.g email links, set editors, or read only access (Dropbox kicks arse in this department)
  • Backup some crucial stuff (nothing huge but)

Pitfalls :

  • The Linux client for Google Drive sucks great fuzzy balls of crapacity
  • Dropbox aint cheap
  • Google Drive PAID is more expensive (per KB) than Dropbox - and - my gmail inbox is eating 11 GB of my free 15 GB (and they deliberately make it HARD and obfuscated to do housekeeping, e.g. I can’t go into old emails and delete the attachments - I don’t want to delete those old emails going back 10 years or more!).
  • The Linux iCloud client sucks MORE fuzzy balls even than the one for GDrive! It’s a shonky appimage or snap or whatever, and all it is, is a wrapper / app that shows the iCloud web page.
  • I haven’t tried pCloud…
  • Not everyone has armhf / arm64 Linux binaries (e.g. Dropbox doesn’t)
  • I am EXTREMELY skeptical of services like pCloud that offer “lifetime” - my arse! Hmmm - that’s probably enough of a disincentive for me to avoid them.

Just remembered something Dropbox can do, that ResilioSync can’t - manage UNIX file attributes, e.g. if I want a file on my RSL (resilio) share to be 0600 (i.e. visible/editable only to me, but not executable) - I have to do that ON EVERY copy of that file on every SHARE on EVERY computer! On Dropbox if I make file “X” 0755, on one computer, it gets that on EVERY other Linux computer…

So - here’s the plan…

While I’m continuing to be rewarded for my blood sweat and tears by getting paid by my employer, and, since I’ve given up smoking for the last 4 months (that’s like a $55 AUD a day habit between me and the missus!**) - I reckon I’ll re-re-subscribe to Dropbox Plus (I recently re-subscribed, then unsubscribed - hence “re-re-subscribe”). Note also - despite (or in spite of) Dropbox announcing they were dropping support for running the Linux client in anything but ext4 - they’ve since gone back and you can run it on a bunch of filesystems : like ZFS, or butterFS or whatever :
Dropbox Brings Back Support For ZFS, XFS, Btrfs And eCryptFS On Linux - Slashdot.

** swear words : just writing that gave me a craving for a cigarette, not kidding anyone, I miss them and enjoyed smoking most of the time (except maybe after a bender on the booze, the next day).

2 Likes

I haven’t got the perfect cloud storage solution either yet, however I just want to congratulate you on:

Wish, I could get to that. I don’t even enjoy smoking - it’s just killing me slowly at 300-500 CHF/month.

On cloud storage, I use Mega and Google-Drive for different types of files, whilst otherwise relying on external physical drives at home for backup (not much use in case of e.g. a fire).

If I managed to save the money for cigarettes, I would probably go for a corporate style solution: My own server at a data centre.

1 Like

@daniel.m.tripp

Thanks for bringing this up, now I will add my own recent experience here.

I was in a dilemma.

A couple of years ago I bought a Christmas Special Offer version of a VPS, that is quite different from the usual VPS offers that are offered by the provider I prefer to use. It’s one that generally has more overall power, but in turn has other things missing, like e.g. a decently sized storage medium. It only has 80GB, if I remember correctly.
After using that VPS for so much time, I eventually hit that 78GB used disk space mark. Well, luckily it’s not like a real disk, where 90% used space is already very critical.
Still – it was clear, I needed an expansion.
So the first thing, that usually comes to mind is just plugging in another disk into one of my Raspberry Pis and mount that in the VPS. Sure, sounds great and easy in theory, but in practice it won’t suffice my expectations.
The point of using a VPS is its nearly indisruptable availability. Since I started the VPS a couple years ago, when I bought it, it literally has never rebooted ever. It’s running since then and it’s running strong.
Whole different story with my Raspberry Pis. There is a power outage about once a year. Things need to be set up tediously again to work and be available. Then, I sometimes screw up internet stuff, e.g. due to the re-plugging of the router or something like that (my previous router was quite unstable, after using it for so many years).
Other times I do something “harmless” on my Raspberry Pi and then I suddenly need to restore a backup.
So, the point is, it seems easy and cheap at first, but on the second thought, it’s just not as reliable and I don’t want to freak out with high availability shit or whatever, just to have a bit more storage on my VPS. It’s just not worth it, for me.
Additionally, even if I would deal with all that tediousness and annoyance, I still would have an issue, which I can’t control: internet connection. Even if I do everything the best way possible, I cannot stop the internet to disconnect from time to time. It’s just how private internet for personal use works. Not to mention the whole dynamic DNS situation…

So, long story short, I decided I needed reliable online storage, that is 99.9999% of the time reliably online and available. It does not have to be fast. I’m patient with such things. I’m also very patient with down- and uploads. I just need the damn reliability. If I want to download something from my VPS, I want it now and not after I get home and fix the server crap, or whatever.

I’ve been looking and looking for online storage. All I wanted is at least 1TB online storage! Nothing special! I literally just want online storage that just is available all the time. I don’t care about speed or how many cores and RAM the storage provides.

The first offer I’ve looked at is at the Storage VPS offers from the provider I’m already on, anyway. I was shocked by the prices, because the cheapest 1TB option literally costs 3 times more than my current VPS solution, just because it has more storage.
Of course, this is absolutely ridiculous and I was under the impression that was just a fluke for whatever reason. Perhaps they just weren’t interested in selling this, so they put up the price intentionally high.

Then, of course, I’ve looked at my favourite cloud storage provider – MEGA.
I love this provider, as I already pointed out many times in this forum (sorry for that), but, as I said, I just need more space. No fancy shit. No features. I literally just need a bucket to throw in digital data. That’s it!

I find the prices MEGA offers its services for, fairly reasonable. At least, if you are fully using all the encryption stuff and all the features and all the magnificient bells and whistles MEGA has to offer. I fully understand the pricing and if I would use all that, I would certainly pay for it. But… I don’t need it. I just need more storage. That’s literally it.

So, I’ve been looking all around the internet for all kinds of storage and it annoyed the crap out of me, that all services advertised features of this and that, when I just needed more storage! I do not care if your storage VPS has 4 CPU cores! I JUST NEED MORE STORAGE!

Now, the only services that are obvious to IT people and yet have no consumer-focused bells and whistles are all the S3 like bucket solutions, like all the Amazon AWS clones. So, this would have been pretty nice… Except.

They are for professional use mostly and therefore have only prices in the professional range. I won’t pay hundreds of bucks a year for 1TB storage. No way.

I was pretty desperate, to be honest, and I was indeed reconsidering to use my own storage, because I just didn’t seem to find an online storage provider of any kind, that just gives me cheap storage at a reasonable price, without bells and whistles I would otherwise need to pay for, even though I wouldn’t even ever use them in the first place.

Now, after all this time, I found something.

https://icedrive.net/

You have to really look for the pricing to be able to find it (at least some time ago that was the case):
https://icedrive.net/plans

First thing I’ve checked were the Trustpilot reviews about this company’s service. Literally all the people leaving negative reviews didn’t know what they were doing and they were all at fault themselves, in my opinion. It usually went like this “I didn’t know how this storage thing works, so I still used it, then it broke. You are dumb!”.

Second thing is the pricing…

They have reasonably priced LIFETIME plans! Pay once, keep using it.
This is precisely my thing. I don’t like renting shit, except there is a good reason for it, like e.g. renting a VPS.

I wondered the whole time, how they are dealing with a lifetime plan, but it turns out a lifetime plan is basically like renting it for at least 3 years. So, I guess, their calculations resulted from a reasonable train of thought. Perhaps they will increase the prices years later, for new customers, anyway.

This provider seemed to offer everything I wanted:
A cheap online storage solution, without bells and whistles I would need to pay for. It’s literally just for uploading and downloading stuff. Exactly what I wanted.

Now the only doubt remaining was the accessability.
I wasn’t sure if I was able to use the service in a GUI-less CLI-only environment. Their app spoke like “I’m GUI only” and they didn’t seem to have a CLI solution, like e.g. MEGA offers the excellent MEGA CLI.

So, before buying, I searched and searched but didn’t find a way. I was already doubting, if I have to drop this one, but…

https://icedrive.net/help/account/does-icedrive-support-webdav

Boom, we get accessilibity options through an established standard.

I’ve never used WebDAV really that much before, so it was a bit of a new place for me.

Setting it up the first time with RClone was very annoying, since I ran into a lot of errors. Almost all the errors are simply because of the fact that RClone’s documentation is lacking tons of information. Sure, if you know how to use RClone and are familiar with it anyway, the documentation is enough. However, if you are new to using it and aren’t even familiar with how WebDAV works with RClone, you can run into trouble, especially since most people do not use WebDAV with RClone, as they usually use one of the pre-built-in solutions for different cloud providers.

So, that’s actually the major part of the story, but I won’t iterate over all the details of how I finally got it to work.

Let me just show you the result of my work:

Apparently, nobody uses Icedrive through WebDAV. I feel like I’m a pioneer with Icedrive.
This was one of the motivations to build a solution like the above one.

2 Likes

What about a self hosted Seafile? It does what you need, at least for me it works exceptionally well.

I’m already, running selfhosted ResilioSync…

It’s completely 100% p2p (it uses bit torrent protocol - the first iteration of the product was called “btsync”) - but I have one machine I consider “the master” or “source of truth” - that’s ResilioSync (RSL) running in a FreeBSD jail on my FreeNAS NAS. I punched a dirty great hole in my firewall (exaggerating a tad :smiley: ) to let a certain port get forwarded to the receiving port that RSL peers listen on - which goes to my FreeBSD jail. I have clients running on :

  1. MacOS - Macbook Pro M1 - MacOS 11.4/5 or whatever
  2. Linux - Ubuntu 21.04 running on x86_64
  3. Linux - Ubuntu 20.04 running on x86_64
  4. Linux - Ubuntu 21.04 running on arm64 x 1 (RPi4B)
  5. Linux - Raspbian Buster on arm64 x 1 (RPi4B) - my pi-hole, OpenVPN and SSH jumphost
  6. Linux - Raspbian Stretch on armhf x 1 (RPi3b) - my TVHeadend server
  7. Linux - Raspbian Buster on armhf x 1 (RPi ZeroW)
  8. Windows 10 - Dell laptop on my desk at work
  9. iPad Pro 12.9 iPadOS 14.x
  10. Galaxy S9+ phone Android 10
  11. Linux - Armbian / Ubuntu 16.04 on arm / sunxi : BananaPi - my transmission-daemon server
  12. FreeBSD : i.e. the master in a jail, but I also have portable version I run as my user off my home drive of the NAS to sync my bash shell scripts

What I need is something “in tandem” to these - that makes it relatively easy to collaborate and share stuff with family, friends and colleagues. RSL aint that easy - sure I can send a link for something and they can maybe get the file - dunno - but - would my MUM know what to do? Could my brother show her what to do?

I can email a RSL link to a new share to anyone I care to - but - when they get the link and try to open it - how will it know to reach from externally, into my internal network, and, it only works if the receiver has the client application installed anyway, even if they were on my LAN. I’d only share this way to other computer geeks, and probably only Linux / UNIX computer geeks…

and @Akito - took a quick look - icecloud looks okay - but - there’s no snap for arm64 (not that there’s an arm64 app for Dropbox either)… Your webdav solution looks interesting… Might be tempted… but right now - Dropbox is too easy - ideally what I’d like from Dropbox is half the storage, for half the price… Note : most solutions for using Dropbox on RPi seem to be using webdav… Years ago - I used to use unison to push/pull sync data, but it was cumbersome…

1 Like

It needs another hole in the firewall/port forward.
TCP port 8000,8082,443 (if deployed with ssl) needs to be open for the sync client to work (available for Win/Mac/Linux). For Android I use Foldersync via WebDAV.

Sharing to outside is as easy as

https://ungvar44.dynu.net/seafile/f/0c5bb067152e46f18fbe/?dl=1
(The link will die after 3 days :D)

I can share a library, in which case I send both the upload/download link to the addressee.
Sharing between family members is the most easy:
we have common folders on our computers, that is synced to a common library to the server. On the server the common library is shared to the group “family” with R/W acces. The clients on computers sync the common folder to the common library, so if someone creates a new file, that new file appears on the other computers in less than a minute - assuming the computer is switched on, user logged in, and client running…

It’s possible to integrate Collabora Office -even the community edition- into Seafile so there can be your own online (Libre) office, which I love, in case we need to write/edit the same document the same time.

Seafile runs on my home server, the router forwards required ports to it. I’m on dynamic IP, so I need the DNS service of dynu.com (of course, any dyndns would work, dynu is just my choice).
Whenever my IP changes, DNS records are updated within 5 minutes (ddclient checks this every 5 minutes). Ususally that happens once a week, but it has a random pattern too.

Anyway, I love my Seafile :slight_smile:

Just one note: I have 1000Mbps(dl) / 300Mbps (ul) connection, which works extremely well for me, but I don’t expect to have that bandwidth to Australia :smiley:

Some people in some areas of Australia are fortunate enough to get Gigabit… Not me… our rubbish government realised what a botched job they’d done of their hybrid (I call it a mongrel) technology model, they’ve now had to go back and run fibre to certain suburbs around the nation - BUT NOT MINE! So I’m stuck with RUBBISH VDSL for the forseeable future, and I’m paying for 50/20 but only gettting about 30/8! And there’s nothing the ISP can do about that… What does grate is sometimes my ISP will send me adverts for promo deals, like 100/50 - but - my line CANNOT do 100/50… If I had 10K lying around I could pay for it - but - I can’t justify that expense! Broadband is infrastructure like sewage (they don’t want me shitting into the gutter), it sould be paid for by the government, and I pay to use it.

I do use a dynamic DNS provider, that’s how I can VPN to my OVPN running on my RPi, and SSH to my RPi from the net, but also - how my portforward rule that allows ResilioSync to send from “outside” to RSL running in a jail on my NAS.

In case it cheers you up: I’m pretty sure, this is even worse where I live. It literally has become a running gag among the indigenous people, here. We have one of the worst internet connections on average. People from developing countries with corrupt politicians (not corrupt like your average government, but corrupt like, they meet actual mafia bosses and discuss politics with them) have better internet possibilities than us (no joke, no exaggeration).

And don’t get me started on mobile plans. While Americans laugh their TikTok ass off with their flat rate mobile plan, giving them nearly infinite bandwidth, while paying something like 50 USD per month, we barely get beyond way smaller, stricter limits, especially for that amount of money. Plus, the connectivity here is extremely bad, pretty much anywhere you go, when you are outside of a major city.

1 Like

Call me ignorant, but I can only remember you vaguely hinting at your location without actually stating it :smiley: … Maybe I missed that post?

I pay $30 a month for my phone plan, which includes ~30 GB data… I’ve never come close to that… and work actually re-imburses me anyway, 'cause I must have a phone for my job…

My VSDL contract with my ISP costs $69 a month, and has no quota, no traffic shaping, no port blocking… It’s not their fault my copper connection is 19th century tech, it’s the government and the major telco (formerly / mostly government owned) which owns the copper… I’d highly recommend my ISP…

@Akito , @daniel.m.tripp will you hate me, if I tell I pay 3100 HUF for my ISP, which is approx. 14 AUD, or 9 €, or 10 USD?
(Sadly, this is not the average, but I’m in one of the lucky areas where this ISP provides…)

Now I’m surprised!
A local measure of speedtest.net:
Speedtest by Ookla - The Global Broadband Speed Test
And another to Australia (Exetel/Perth):
Speedtest by Ookla - The Global Broadband Speed Test
I expected a lot less bandwidth… :open_mouth:

1 Like

True…

However, just a few km south, I have a 10Gbit fibre optic internet connection at home (not much use yet, as my ethernet adapter only manages 1Gbit), 5G coverage >95% of the country (including mountains) and mobile plans with (really) unlimited internet start at 23CHF (about 21€).

But before you start crying, think that rent and food are at least twice as expensive in Switzerland than in Germany.

1 Like

Well… There’s me, earlier this week singing the praises of Resilio Sync - must be that Murphy bloke comin’ along and f–king me over!

Earlier today, I’m trying out Kali 21.x on one of my Pi4’s - booting off SD-Card (can’t get it to boot properly off USB) - and I decide to sync up my main RSL share - where I keep all my VITAL mission CRITICAL shell scripts…

But it won’t play ball, on the web client I see “database error” but it won’t tell me what the F–KING error is… So I “apt purge” that sucker out of there, reboot, apt install again, from scratch, and I get the same thing again, EXACT same thing “database error”… so I give up, power off the Pi running Kali and carry on what I was doing before…

So - anyway - I’m also working at the same time, and I try to fire up another SSH session on a customer Linux jumphost, and type “ssh jump” and NOTHING happens (zsh doesn’t know how to finish my sentence)… So I look in .ssh/config, and then .ssh/customername - and - BOTH files are ZERO bytes! They’re actually symlinks in one of my RSL shared folders… so I look in ResilioSync and that share is RIDDLED with zero byte size files that USED to have DATA in them!

WHAT A PIECE OF SHIT! Dog’s BREAKFAST! UNACCEPTABLE!

And - it metastasized this cancerous BLIGHT to EVERY other computer that was peering that shared folder…

I might have to start researching an alternative… Last time I tried something like the NextCloud seemed the most promising…

I now have to go to 10-13 computers and manually locate the “archived” versions of these files (each share has a “.sync/Archive” folder) and restore them - MANUALLY…

image

And - it gets worse! Sure it backs up files, I can recover from this vendor bug caused mini-disaster - BUT - I shouldn’t have to - and - in many cases, the “backup” of the file doesn’t have the same name, it will ahve a 1 or 2 or 3 appended (or inserted, or prefixed, in the case of files starting with a dot - e.g. .SHITFILE becomes 1.SHITFILE and even 2.SHITFILE, in the “Archive” backup)…

2 Likes

I’m FUCKING ROPABLE (that’s like irate on steroids) - 2 hours work to get everything back… and I’m still unsure - not 100% confident…

I should send them an invoice, most of this recovery work was done after my usual working hours, I charge my employer time and a half for such work, and they bill customers maybe $200+ an hour for "After Hours shit on a weeknight (it’s double time on a weekend!).

AND HOW DO I EVER TRUST THEIR PRODUCT EVER AGAIN???

My trust and confidence in their product has been SHATTERED!

Plus - when I first noticed this I was in the middle of installing Xymon / Hobbit on some Red Hat servers and got sidetracked - was supposed to finish this shit at 5pm and it’s now 9pm and I’m still working!

I’m on a bombing run tonight…

Anyone know of a self hosting cloud sync product that has CROSS platform client binaries?

i.e.

  • Linux on i686, x86_64, arm64, armhf, arm7l
  • MacOS on x86_64 - and - Apple M1 Silicon
  • Windows 10 on x86_64
  • FreeBSD on x86_64
  • FreeNAS Jail (x86_64) - not a showstopper, but it would be nice…

I just did a quick search, and its seems there’s still no arm (e.g. Raspberry Pi) client binaries for NextCloud…

I took a look at that Seafile product, and it wouldn’t cut the mustard for me (only comes as a snap for x86_64 and i686 right?)…

If MegaSync does support those more obscure platforms - I may re-evaluate…


there’s syncthing, which has a plugin for my NAS - and - seems to be multiplatform… hmm - it’s written in GoLang!!!

I’m going to give it a bit of a test drive - I’ll never trust or rely on Resilio Sync again… fuck me over once, consider me wise…

Last time I checked, it was open source. So, in the worst case, you would most likely be able to compile it to fit on any platform.

No. I never ever would use snaps as I hate tham like flatpaks.
https://help.seafile.com/syncing_client/install_linux_client/
Here it tells the tale how to install the client via .deb files to Debian / Ubuntu :slight_smile:
However, Debian provides Seafile client in its repo, not the most fresh though…

Installing the server is a different thing, there’s no install package.
https://manual.seafile.com/deploy/
Install dependencies with apt-get… then run the install script.

Being inherently lazy - took a few looks at syncthing, and decided I can’t be arsed learning a new product…

Consider me warned…

Here’s what I’ll do in future :

if it’s a non-standard platform (i.e. not Linux on intel / AMD) then I’ll first sync a share in read only mode - so garbled shite on a new system can’t shit all over everything else…

By non-standard - I’ll consider Kali on ARM and Raspbian on RPi Zero as “non-standard” because getting RSL working on an RPi ZeroW is DIABOLICAL - it seems that the Raspbian kernel on Pi Zero is not 100% “armhf” but also has bits of “arm7l” and the rslsync binaries don’t like it… I have a workaround for that (the vendor supplied systemd service will NOT start, but I an start an alternative binary as a “portable app” manually…

But I’d have expected a bit better of a company that slugs “pro” customers a usage fee for using “pro” features of their mostly cross-platform, but mostly proprietary, product…

Huhu, I’m using SyncThing for syncing between my Win10, Debian, and Android for years. Flawless. But it’s no Cloud service. It’s only between Clients. There’s nothing to be learned about, except things, you know already, obviously. :slight_smile: