Revisiting PAPERLESS all these years later

After something like seven years, I’m rebuilding my entire organizational system across my Macs, iOS and the Cloud.

After mind mapping what I think it’s going to look like, I’m spending some time with the original Field Guide that started it all for me! (Thanks again @macsparky !)

I’m going deep and plan to spend the next few months going through every file I have (including photos, videos and music, too!) and ensuring that they’re all correctly sorted in the proper home. I’m sure I have a lot of duplicates, too. So I’m looking forward to getting that all cleaned up. I’m keeping it all in a Drobo and in Finder vs. something like Devonthink for now.

I plan to use a few of the Setapp utility apps as well as a few others to speed things up a bit, but if you have any suggestions, I’m all ears!

Cheers!

3 Likes

It feels like that book came out yesterday

2 Likes

I’m looking at reorganising my digital filing and going paperless. I notice this particular field guide was last updated in 2014. Since tech moves on, do you think a lot of it is still useful and relevant?

1 Like

I’m mostly revisiting it for the framework, file naming and file structures, etc.

While I would be first in line if @MacSparky released a new version, I feel like the updates have been in all of the MPU episodes ever since.

Flipping through it made me realize just how much my system is based on what’s in these pages.

Gang … I’ve started outlining an updated version of Paperless. I’ll be starting from scratch and it will be a video product. My current production schedule is:

  1. Photos FG
  2. Mystery Title
  3. Paperless, Second Edition

Would love to know where you need help.

15 Likes

I think a section on managing digital notes would be great

2 Likes

Info on planning for moving files from old formats to new ones, how to do that in bulk efficiently. Goal is to ensure that your paperless archive is readable 10-20-50-100 years into the future (hint, plan to move all files over to new formats before you decommission an old machine or app). Why so long? There isn’t going to be a rosetta stone for old 8 inch floppy drives or old revs of MS Word, think historical archive. :wink:

Talk about machine and operating system independent file naming and organizing schemes.

Talk about efficient pruning systems to delete the truly worthless stuff when it’s past it’s useful lifespan.

4 Likes

OogieM said more gracefully what I have been trying to say.

1 Like

I struggle figuring out where to put different kinds of things. What should be in the iCloud file system vs. what should be in a notes app vs. what would be best in an everything bucket. Then how to get all of that to work together as a system. When I’ve scanned something into good notes and marked it up, should I put it somewhere else or just remember that it’s in good notes.

And how to settle on a system. Get everything working in that system, and then not mess with that system.

2 Likes

I can’t wait… I think a V2 could be the most powerful Field Guide yet. In many ways, it could be the TROJAN HORSE that could connect the other Field Guides and MPU/Automators episodes to tie it all together. It’s in this way that it could have the most “crossover” appeal for civilians, muggles, etc.

But I think there’s also room for a higher-level discussion in its pages aimed at the MPUser specifically around the overarching Design of the ideal, scalable, “system”. A set of guiding principles that anyone could use to build their system regardless of the specifics of hardware, software and even cloud solutions. We can probably expect all of those to be around in the Future, they just may not be called what they’re called today.

What we know for sure, is that we will have many more files than we do today and use much less paper. When I first got PAPERLESS, it was with the thought of “I wonder if this is really possible…”. In 2020, going paperless is like: “Well, yeah… Of course!”… But then people will do it all wrong and this is where V2 would come in… To show them the best “HOW” out there.

I would echo the angle that @OogieM mentioned. I’ve come to realize that the main reason I revisited the book recently was to think about what’s in the first part of section 4:

4.1 = Naming Files
4.2 = Storing Documents

With a solid backup plan, I’ve chosen to go hard on nested folders, tagging and OCR for portability reasons (we’ll see how long I stay out of AI-powered “everything buckets”). I’m focusing more on organizing my system to make it easier to ACCESS/USE my files to avoid ever resenting all the days I spent scanning documents into it.

… For years it felt like I was taking tremendous care capturing and prepping my files only to send them off into space on a rocket, knowing that I’d never see them again. Which just makes you feel silly.

4 Likes

Certainly a full review of the current batch of scanners is in order, including any separate scanning+ocr software. I bought Paperless before I even knew about MPU. Thanks to Paperless I bought into an iX500, Hazel, and PDF Pen Pro and still use them. I just looked at Paperless again and there are lots of software programs discussed which I don’t use. I would guess it all needs to be reviewed and revised.

1 Like

YES, this too would be extremely helpful. Perhaps even going into the styles of systems that fit best with how people think and retrieve. For example, I do not necessarily think in tags. I think in locations, I also have learned that flatter file systems are more efficient for me. But other people do think differently. Ways to test and determine what way you like to retrieve information and then how to design a system that supports your natural tendency and ability would be very helpful.

Yes, It’s no good developing a clean scanning workflow if you can USE the resulting data.

One thing I’m discovering right now is that due to the problems with OCR software decades ago many of the older files I scanned are not really searchable. That influenced my naming system (so I can know the contents by just looking at the filename) and my storage system by putting things in buckets/folders so they are easy to retrieve and limit how many individual items are in a single place so searching is faster.

One current project is how to upgrade those old scans by running them through an OCR process efficiently. I’m still debating/testing which one to use that will fit in well with my existing system. It’s ok for the new stuff to be scanned that way at the beginning but as part of upgrading old formats I really need to also deal with the older stuff so it’s more easily searchable.

I want to see more lifespan/estate stuff. Part of it is wanting to have a clean system that gives access to critical estate documents at the time needed with minimal hassle to those having to deal with my eventual demise. While not exactly part of paperless, since so much of the important information is in electronic files, that needs to be addressed there as well. As laws change, when does a digital will/estate plan become legal? Right now they are all on paper, signed, notarized etc. But in my county property deeds are managed as electronic files once the original is signed and they never store the original paper. Most of my state filings for business documents are entirely digital. I expect that to continue to evolve and our personal systems have to interact and mesh with the forming official ones in an efficient manner. If my property deed is an electronic file at the county courthouse how will my heirs know to go there to get it? If my estate documents are stored in an encrypted file at the estate lawyers how will they get access as the law firm changes and evolves too?

4 Likes

I agree with elements of several previous posts.

I have a decent workflow going for digitizing and organizing my documents (although always on the lookout for tips to enhance my productivity and efficiency) as well as an extremely good backup system.

The one weak link and question mark is my archival system. What are the best media, drives and services to use for protecting data that must be maintained for the long term?

How do we appropriately evaluate new technologies as potential replacements?

How safe is my data on mechanical hard drives that sit on a shelf for several years? Do these disks need to be spun up periodically to maintain their integrity?

What about the so-called “flipped bits” or “disk rot” maladies that are rumored to erode archived data after a few years? Is this fact, fiction or a bit of each?

I feel pretty well situated to quickly and reliably access any document in my system going back at least five years with no problem whatsoever. But for data that is older that that, even though I periodically do tests to recall random data, I don’t have nearly as much confidence in that data…even though it had been transferred onto new 4 TB Seagate HDDs in the past year.

I currently use the Iron Wolf drives in my NAS and RAID 1 drives and Barracuda drives for single disk archive drives. Would you recommend anything different?

2 Likes

I second this and agree with all of @OogieM’s concerns . Maybe @MacSparky you can apply some of your legal expertise to this problem. I tried to discuss best practices for preserving digital estate documents with our estate attorney, who was clueless on the topic.

Off topic, but I’ve long thought there is a rich opportunity for someone to develop a product for estate attorneys and clients to provide an ultra-secure digital vault for estate and legal documentation. Something more specific than sharing a Dropbox folder with the lawyer. My partner and I had an estate attorney to whom we gave a lot of documents in digital form, and then she up and left the firm and we had no idea who had our data.

Katie

2 Likes

Agreed. Now I realize that any specific drive or manufacturer recommendations will be obsolete almost as soon as you say them but at least provide systems and tools for deciding how to determine whihc drives and media are safer.

Another +1 for this. Right now I’m implementing AWS as a potential long term storage. But it’s log learning curve to get it up there working the way I expect. In fact heading over to an all day AWS conference virtually today to try to get up to speed on the tools and terminology. There’s another project for you, a how to use AWS and other cloud storage systems effectively.

A lot depends on the technology of the hard drive. A more mature tech but a new drive will in general be safer. My hardware expert hubby says that it’s a good subject for a masters thesis every 5 or 6 driver generations. :thinking: But in general yes, you need to verify the system works periodically.

Most of the problems happen either with physical degradation of the media or when copying files from one location to another. CDs develop these worms of corrosion that destroy vast swaths of data on them. They are in the layers of the CD and happen even to gold or archival ones. Storage in any compromised location (moisture, salt air, heat etc. will make that happen faster and by the time you can see the corrosion spots you’ve got a nice coaster.

Bit rot seems to be mostly limited to how many times you transfer a file from one location to another. So as you upgrade your devices and move files from one device to another you have to test them. Ideally every time but that gets very slow. Certainly each time you make a change from one major technology to another. I lost several years of data due to failure to verify the transfers. The topic of how to verify accuracy in the copy of data made on backups or archives would be an important part of going paperless and would be really helpful.

5 Likes

+1 for this :point_up_2:

+1 for this :point_up_2:, too

Echoing all the above as well as a list of buckets.

Example, I used to have an Evernote bucket and then from much influence and feedback of MPU forums found that DTPO was much better suited for my needs.

Is there a better bucket? Pros/Cons, etc

I have a solid naming convention that I use for my photos, but one of the big struggles has been proper-file naming conventions for PDFs.

Examples of sample workflows are always a plus, sample naming, categories, etc. The incorporation of apps like Hazel, Keyboard Maestro, etc into all of this as well.

1 Like

Questions from a non power user

I want to know where my data is, and where my family’s data is. More and more stuff seems to be residing primarily on cloud services. How do I manage that? How do I think about back-up in the context of everyone having multiple mobile cloud connected devices that are increasingly also cloud dependent? And what about data that’s tied to particular apps? Especially sensitive data, like you’d keep in 1Password."

Which file formats have the greatest flexibility and longevity? When and how should I think about converting?

When I dig in to start working with files, how can I expect them to behave? How do I “move” files from point A to point B? What is happening under the hood? If I “share” a PDF that’s in document folder to another, does that create another copy of that pdf, or is it a link, and if it’s a link, will it break if I move or rename the “original”? When I create a pdf from my print dialog box and then share it, e.g. to a note, where is it stored?

How do I deal with my downloads folder and other apparent junk or mystery items? How can I tell which files I can safely delete and which I can’t?

What are two or three good naming and organization strategies you’d recommend.

How do I get my paper in and OCR’d these days? I’ve heard about problems with both PDF pen and scansnap.

As a lawyer, can you speak to which documents are generally most important to keep and in what format?

I’d be interested in your current answers to these questions, and I’d be interested in learning more about basic file management.

4 Likes

Been thinking on this. It is a great question.

Document/file naming conventions. I still use year_month_day_file_name.

As a senior, I too am concerned as to how my clean up crew will be able to access my digital documentation after I am gone. They SURE DON’T need to be experts at a specialized app.

Universal file extensions, that LAST through time, (like .txt). And not only documents, but video too. (I have several old videos of my kids, that I can no longer play :frowning:).

WHERE and HOW to do long term digital storage, so that not only can it be accessed by my device, but again, my clean up crew when I am gone.

Is there a way, using iCloud or Dropbox, to store my files on device, AND in the cloud, AND they not automatically delete/overwrite the other.

A side note:
I live in hurricane land. Having my “stuff” in digital format, AND accessible when there is no internet is a MUST. The days of carrying a plastic tote with your important documents in it is about gone. Most shelters don’t like them. Security.

3 Likes

Replying to both previous posts. I’ve given lots of thought to all of this and have files dating back 40 or more years! So some thoughts:

  • Storage is cheap. Save everything, because you never know if you might want or need something.
  • Because storage is cheap, don’t bother with compressors (like ZIP).
  • Luckily formats have been settling down. There was a time when plain ASCII was the only safe choice for text. Now I have no concern about using PDF since it’s be around for decades now. JPEG and MP3 will always be with us. And there seem to be plenty of movie format converters around.
  • If you have to save as source, it seems like only Microsoft Office is “safe”. I’ve been burned by many others. But if you don’t need to go to source, PDF is generally great, although I’ve used CSV for spreadsheet data.
  • Mac’s Finder and Spotlight (especially within Finder) have solved all my searching needs. Files kept in a folder hierarchy. Easily move between Mac and Windows and Linux.
  • A cloud syncing service like DropBox or iCloud is not a backup. Anything that syncs basically means you only have one copy. You need a cloud backup service if you want to back up to the cloud. My primary storage is on my home server computer with cloud as well as clone backups (clones saved in a bank vault).
  • I’ve got a document with instructions for my executors listing all my passwords (actually just login and 1Password passwords are necessary) and how and where everything is stored. Some services cut off access upon death. I don’t use them for anything of value. It does help that I’ve got computer savvy heirs.
5 Likes