Escaping the iCloud roach motel

I thought CloudKit sync is a lot more robust than simple iCloud sync. This does not explain many issues mentioned in the Update v2: Needing Serious Advice re: Losing Faith in Apple Notes discussion

Sorry, I know next to nothing about how iCloud works and even less about Cloudkit. I have on occasion offered ā€œsolutionsā€ that appeared to work for me, at least once.

The article that @R2D2 posted in ā€œ Best Option . . . ā€œ is one of the most informative I have read. I had no idea that iCloud is a database.

It states, "iCloud is one of those features in macOS which works almost all the time for the great majority of users.ā€ I agree, and thatā€™s why I donā€™t depend on it.

Dang this thread is informative. A heartfelt thank you to all

3 Likes

Per user, right?

Ah yes, I remember getting to the bottom of that and being unwilling to hit that big off switch, given I was already confident it would not work. (Because you canā€™t get a bigger off switch than wiping and selling the laptop and buying a new one and it still has the same problem.)

My case proved that sometimes itā€™s really not you and there is nothing you can do about it.

All I can offer is that I was exhilirated when mine started working 842 days after I first reported the issue.

With DEVON, one thing is where you database is stored and other thing is the location of the log data needed to sync between different databases on different devices (thatā€™s the ā€œsync locationā€). The sync location is not a DEVONthink database, itā€™s more log of changes. DEVONthink does not have the concept of a ā€œunique databaseā€, every device has its own database on a given local storage and the sync location receives and propagates the changes to the other devices. If there are conflicts, they are resolved locally on each device, in a quick and consistent way. This is a hard problem.

To add some added difficulty, the DEVONthink technical burden is that it predates the era of cloud syncing, and they have to support legacy methods that their user base has come to depend over the years: I canā€™t remember if I ever used Bonjour or WebDAV since 2003 on any of my Macs. Ever. Yet DEVON supports it and this is probably the reason why they cannot do what EagleFiler, KeepIt! or Notebooks can easily do: just store everything on iCloud Drive and rely on iCloud syncing altogether. Obsidian comes also close, but due to the way Obsidian access its files, to have your files on the Obsidian iOS app, your vaults must be on the dedicated iCloud Obsidian location.

So, to some it up: never store your local .dtBase2 packages on a cloud folder, wherever it may be. Use the syncing method you like (CloudKit worked well and it was fast for me). But also never index iCloud Drive folders or files into your DEVON databases, specially if they have syncing set-up, thatā€™s asking for trouble when iCloud starts ā€œoptimizing storageā€ and other funky things.

3 Likes

Fair. As was mine sans emoji. All Iā€™m saying is that Iā€™ve been close to some of these issues in my line of work. The prevailing thought is that these commercial clouds with government certifications are ā€œsuper secureā€; not true. Generally speaking theyā€™re the same environment(s). And having a certification isnā€™t relevant if your data isnā€™t hosted in said environments; ie IL4. Different teams, pricing, business units, support, etc. Could go on but you get the pointā€¦

There is a reason apple canā€™t sort this. These are their essential apps listed today on my iphone. Iā€™m surprised they get any work done at all. :grin:

4 Likes

Yes. Thatā€™s the price for a Business Standard Google Workspace account. Business Starter with 30GB is $6.

Storage that can be shared with up to 5 others is available but I donā€™t know how it works.

:scream_cat:I hope I wonā€™t have to wait that long!

I have reluctantly concluded that I will only use applications that can export my files in plain text, MD, RT and PDF formats. I no longer feel secure putting important information in AN since I canā€™t trust that the synchronization will stay reliable. I also donā€™t want to depend on third party apps like Exporter to export my notes. One never knows when it, and similar apps, may be abandoned.

2 Likes

Iā€™ve never had any issues with iCloud Drive, but for what itā€™s worth, when I decided to stop paying for Dropbox, I moved to Synology Drive and that has worked very nicely for simple file-syncing: the kind of thing we used to use Dropbox for.

My wife and I have several Macs - about three each - syncing one personal folder each and one shared ā€˜teamā€™ folder between us. The iOS clients mean I can access everything with minimal effort from my iPhone and iPad too - the folders appear in the iOS Files utility, for example.

I have one on-site and one off-site Synology Diskstation, and I can arrange whatever kind of backups I like from either of them. You do, of course, need to have a NAS in the first place, but you then get however much storage you want very cheaply.

Just my tuppence-worthā€¦

Welcome to the fold, my friend.

Soon you may conclude that exporting isnā€™t enough ā€“ what if things go wrong just before you were about to export the most recent changes?

<Darth Vader voice>Come join the plain text cult, @Bmosbacker!</dvv>

Seriously, after maybe a year, I never looked back, and that was more than a decade ago. Nowadays, you can do almost anything in plain text, with only mild sacrifices: simple charts, tables, slide decks ā€“ Iā€™ve even found a timeline app, though I havenā€™t tried it yet.

Happy to expound if itā€™s ever helpful!

4 Likes

The last time (about 5 years ago) I looked Microsoft and Google had a ton of white papers including some on how they store & protect customer data. If you really want to dig into the subject I found a 74 minute tech talk from 2013:

How Google Backs Up the Internet

2 Likes

I obviously canā€™t say authoritatively, but I think it varies across providers. For example, in the past GCP leveraged sharding and couldnā€™t guarantee that data remained inside certain ā€œboundariesā€. This was a big problem for some of their larger customers. This was a few years ago and happened prior to some of their recent certifications, but was a huge issue and I believe caused them to at change their stance.

Depending on classification and compliance some clouds are more cloudy (technical term) than others. Some of these clouds are just managed data centers that offer modern devops tools and have cloud consumption models.

Sticking with the Google theme, theyā€™ve been really good about publishing material on SRE, autonomic security, etc. and I think they do redundancy well. Microsoft is much less transparent in my opinion, but also had a different strategy to market. All I know for certain is that these systems are extremely complex; this will lead to failures of greater impact even if infrequently. Again, a lot I donā€™t know and I can only make assumptions based on what Iā€™ve seen/experienced.

I so disagree with this. All the charts, styling, timeline, linking, etc, are not plain text. They may be using plain text files as their input, but their output is anything but plain text, because plain text does not do those things.

Using plain text sacrifices everything, every bit of processing needed to do something with the text. I see no point in an obsession with plain text when in reality everyone wants rich text with images links and charts and timelines. I appreciate that there are coders out that can do all this and that there are apps that will manipulate plain text, but there is little difference between that and a word-processor it apps like Craft and Bike. Nearly every app will output plain text at a very minimum.

We are in an age where apps do amazing things and people want to stay in a 1980ā€™s text format. Iā€™m so glad for apps like Bike, Craft and Tinderbox.

No offence intended, but having followed the ā€œplain textā€ hype for five years and found it seriously lacking in nearly every area of tech life and looking for endless workarounds, Iā€™m so glad that we have a wealth of rich text apps.

3 Likes

Weā€™ll have to disagree.

When I make a Mermaid.js chart, I have the data I need to recreate it in some other app, by hand if necessary, even if every instance of Mermaidā€™s interpreter vanishes. I donā€™t have to dig through a bunch of XML or figure out how to extract it from some database.

If I make a table in MultiMarkdown, it can be rendered nicely by an app, but I can still read it in the plain text file, if necessary with the addition of some spaces.

If I save my structured data in CSV, I can use it in an SQL database, a spreadsheet or any number of other apps, without fundamentally altering it.

My goal in using plain text is not to replicate what full-featured apps do, but rather to separate my data from the apps I use to access it, so that I never again have to struggle to get at my data because some app or format has died or changed.

The apps I choose to use are very good at making my data look and function how I want ā€“ thatā€™s why I use them ā€“ but they donā€™t hide or obfuscate that data in ways that are going to complicate my life later. Thatā€™s my goal.

Do I want the beautiful formatting and useful functionality all the cool apps offer? Absolutely, where I can get it. But Iā€™m not willing to sacrifice the accessibility of my data for it.

And, thanks to the ingenuity of some independent developers, I mostly donā€™t have to. I can have my plain text data and use it too.

To this end, Iā€™m selective about what plain-text formats and apps and bells and whistles I adopt. I donā€™t use anything like Obsidian data view; I mostly use apps that let me render multiple formats of markup syntax.

And I admit Iā€™m very fortunate not to do a lot with video or other complex visuals (though I do some of the latter). When collaborating I often have to shift to apps that others can use simultaneously, which takes me out of my plain-text world at least for a time. But thatā€™s OK, because for the most part the inputs to those processes are still where I can find them, in plain text.

4 Likes

This argument comes up a lot. What if Microsoft Word vanishes or Excel or ā€œXā€? This is not a real argument. Nearly all apps allow you to export your data. From plain text to RTF, PDF, csv, whatever.

No app disappears that quickly that I cannot get my data out. I have no fear of any app that I use vanishing overnight leaving my data locked in. Thatā€™s just not going to happen. If Microsoft shut down, I could still export my data to RTF and not need to search through an XML.

Besides this many apps like Word can be opened in open office or pages or other apps.

Using rich text apps does not compromise your data being accessible in any way.

1 Like

I have it on good authority that none of us need to worry about that.

Isnā€™t the a minimum number of accounts, or am I misremembering?

No. I have a business starter Google Workspace account and pay $6/month for one user (me).

I can add or drop users at any time and ā€œOn your next monthā€™s billing date, Google charges you for the total number of active Google Workspace users at $6 USD per Business Starterā€

1 Like

It can be hard though, and labor intensive. Think about people using Claris Works, or Microsoft Works. Or people who want to move from Evernote.

The only way I know to move data from Evernote with a minimum of hard labor is to buy and use EagleFiler, as a half-way point, not a destination.

If you use Bookends or Endnote or Zotero as a reference manager and have to move your data, export is non trivial.

1 Like