DEVONthink: stuck at the bottom of the hill

This is a bit where I’m at. My finder tends to be a mess and I’d like to fix it. Toying with importing stuff into DevonThink to manage it easier but also toying with storage space. I think too tagging will be key for me to organize things within DevonThink. Still like folders but tagging provides greater flexibility.

I took the plunge and jumped in pushing through DTPO to test if it can truly replace Evernote for me (or at least bring Evernote to a free plan). I caved in and got the iOS apps even though I wasn’t sure if it would for my configuration, but I needed a way to test the system out, when I go out.

Also based on the advice from many of the replies, I am about 3/4 of the way through the book “Taking Control of Getting Started with DevonThink 2 by Joe Kissell”, that book has been tremendous help in getting started and understanding terminology more. I am trying to approach DTPO without really thinking about Evernote, but more of just “what is my workflow and what do I need to get done”.

Started off with a “Testing” database until I was comfortable with it and now starting to import my Evernote notes.

My thoughts so far in regards to DTPO workflow.

  • Using a single database since much of my life overlaps and until I am comfortable enough where I need separate ones. I don’t recall if I saw a way for the AI to search properly and make connections across databases, so for now it’s a single database.
  • I have the Database stored in UserName/DevonThinkDB/ (instead of the ~Documents folder)
  • The Database syncs to Box (I was in debate of having it at Dropbox, iCloud, or my Synology but couldn’t find information if there were any differences that users felt in sync speed).

Current think I am trying to understand :thinking: is importing notes from Evernote. I am still doing testing in that Testing database, but here are those questions.

Importing Notes With Attachments from Evernote Questions 1

  • Is it better to import? Or should I just make new notes by copying each one individually.
  • I noticed that when I import an Evernote notebook, it imports the notes which is great, but if a specific note has a PDF attached or an audio recording attached, it creates a separate folder/group out of it.

Example if I am transferring “!nbox” (yes that’s how I name, so it stays on top and brings to focus). There are 80 notes in there, 2 of the notes have PDFs or Recordings.

After importing 1 notebook, my structure looks like in DTPO
Folder !nbox
Sub Folder within !nbox that has the name of the note and contains the note and PDF/recording.

Importing Notes With Attachments from Evernote Questions 2

  • Is this the behavior that happens when importing notes that have attachments?
  • What are “best practices” that you have experienced for this type of import/storage?

Storage/Sync

  • Do you recommend a particular sync? (Dropbox, Box, iCloud, or utilize my Synology) My concern is accessibility and fast syncing.

Thanks!

1 Like

Over the last year I have tried five ways of syncing:

  1. Dropbox: Fast but I quickly grew out of the free amount of space, but didn’t need the full paid option.

  2. Webdav: For $5/month I created a webdav server on Digital Ocean with plenty of space. Also fast, but it took quite a bit of work for me to set up. Wasn’t sure about how to maintain a webdav server.

  3. iCloud: I was a beta tester and this is one of the sync options I settled with. I like the plans apple offers. No need to pay for more space than I need. It is also fast. I view having multiple databases in DT as a way to set up selective sync with iCloud. I could pick and choose which device gets each database.

  4. Bonjour: This is a direction connect between devices. Wire and/or Wireless. I often work without access to the internet and this works well. Fastest of all the sync methods when using wifi or a wired between devices. Kinda slow when using bluetooth to connect the devices. BTW, with this method you select one of your devices to be the host or base device. You don’t have to, but that worked well for me.

  5. Box: For me, this was the most problematic. I had more issues where their service didn’t seem to be available. When it worked, it worked fine.

The great thing about DT’s sync: you can set up redundant syncs. You can sync the same database using multiple methods. Which ever of the methods I used above, I always have Bonjour setup as an option for when I don’t have the internet. I also set up redundant sync methods during testing over the last year.

I have about 10 databases syncing 11gig of data (1,000s of pdfs, markdown, movies, etc…)

Remember, never place your databases directly in a cloud storage space. You want to keep the database locally, and let DT’s sync options do the syncing.

4 Likes

Thank you for this advice. I’ve been using dropbox but considering switching to iCloud, now that DT has optimized for it.

This advice is one of the main reasons I have separate databases for some things:

“I view having multiple databases in DT as a way to set up selective sync with iCloud. I could pick and choose which device gets each database.”

That is true of any way of syncing, not just iCloud.

And this is also important advice, has been mentioned before, but bears repeating:

Remember, never place your databases directly in a cloud storage space. You want to keep the database locally, and let DT’s sync options do the syncing.

I guess it just didn’t work for me or I didn’t get it. I feel spotlight works fine for searching through my files. If you want better searching there are options that will use the current file system (Houdaspot, Alfred, Path Finder). When I used devonthink, the thing that bugged me most was that I couldn’t save directly to a folder from the save dialogue box and getting to a file to attach it to an email was also tricky. People overuse this saying, but I really do think this is a piece of software that you either love or hate. You have to try it to see if it works for you. I might give it another try just for my PDFs.

I understand that I shouldn’t place a database file on a cloud, but what if I want to work with multiple TBs of data in DTPO? Should I just index those files?

Curious about what you meant by the following:

“When I used devonthink, the thing that bugged me most was that I couldn’t save directly to a folder from the save dialogue box…”

Do you mean that when saving a file, you couldn’t specify the DTPO group/folder that you wanted the file to go into?

Regarding this > “and getting to a file to attach it to an email was also tricky” > what was the challenge?
Right clicking on a file, allows one to ‘send by email’, which opens a new message in your default email client, with the file attached.
Granted, I also use “Yoink”, and simply drag&drop several files onto it, but drag&drop directly into a mail message also works.

That all said, I completely agree with you in how Houdahspot/Alfred etc., can do the job as well, depending on one’s needs.

1 Like

I initially felt the same way about the searching. I even returned the software. For me, my job changed. I moved to a research role that required working with thousands of PDFs and searching through them in a more intelligent way. DT gave me the answers I needed better than the others. I also like DT better than the other options when I had to search across file types. I keep my notes in markdown, my own papers in LaTeX, and all my references are in PDFs.

My other issues is that I no longer wanted everything in iCloud to sync to all my devices. DT’s sync makes that very easy. You can either break things up by database and only sync certain ones to each device or you can download on demand.

DT has a directory that you can save files to and it brings them into the database automatically. I dragged that directory to my favorites in finder and now it shows up in all save dialogues.

I think the biggest issue/limitation/hurdle with DT is its documentation. I honestly needed something that would walk me through how to do things. I think the included documentation assumes a certain level of knowledge that I didn’t have.

I learned by experimenting because I had the absolute need for what DT offered. When I first purchased the software I was turned off. I found it very difficult to set it up the way I wanted, or even to wrap my head around the sync options. I learned to lean heavily on their forums. They are very helpful.

2 Likes

Do you have any sample DTPO folder scripts as you mentioned here?

Hi Brad.

Yes, it bugged me that that I couldn’t specify the exact save location. I don’t remember exactly what else I had issues with if attaching a file to an email is no issue. I remember running into annoyances that had to do with how devonthink sections off databases from your files in finder on MacOS. I do vaguely remember devonthink not liking it when I opened file and renamed it by clicking on the name along the top of the window.

I probably need to go back and give devonthink another try.

Under the main Devonthink menu, scroll down to “Install Add-ons”, and have a look there.
Check if you have the “Folder Action Scripts” box selected (and while you are there - check the others, I have them all selected).

This will then allow you to install/use the Folder Actions, under the ‘Services’ menu, that appears when you right-click on a Folder in Finder.

Hope the above makes sense - shout if I can clarify.

That said, have a look here for more details on further options (hope they help!).

Folder Action 1 - using KM

Folder Action 2 - AppleScript

Folder Action 3 - Automator option

Edit: Wanted to add the Applescript content from one of my folder actions here, but cannot work out how to insert it as {{code}}, whilst maintaining the formatting/indenting?
Any suggestions @RosemaryOrchard

4 Likes

Ah - yes, I can see how that would be annoying.

Maybe two points:

1.) There are a multitude of ways to get things into Devonthink, and honestly would recommend that – assuming you want to make a serious go of it – you spend a bit of time on the Forums/with the manual, learning about the various options.

Depending on what approach you use, the file will either be dropped into the so-called Global Inbox (which is at the very apex, and requires you to manually feed it into a specific DB), or you can be prompted in what group you want the file, or you can drop it on specific folders/drop-points on your system, that then have folder-actions running, and feeding those files to specific groups.

Of course, the beauty of it all, is that you can run a variety of these, or all of them, which will cover just about any variation to getting files in.

Off the top of my head, I use the following:

a.) Clip from Safari, using the toolbar droplet/shortcut key – asked where I want to place the file;
b.) Using the Devonthink ‘Sorter’ window, either dropping the file into a particular Inbox of a DB, or a particular folder/group of a particular DB;
c.) Mail rules inside Mail.app to Devonthink - sends mail that matches the necessary criteria to DTPO;
d.) Drag and drop into DTPO or onto the Dock item (my dock is hidden, so I seldom do this, but have at times);
e.) Using the Finder Sidebar Inbox location, most frequently in conjunction with Default Folder X; and
f.) Numerous Finder folders (including Dropbox etc.) that are either being Indexed, or have a Folder Action associated with it, to import the files.

I think the above covers all of them, but there might be more glaringly obvious examples I am missing!

2.) On your point about how DTPO doesn’t like your changing of files, when opened externally – that would suggest you were importing your files into the DTPO database. To be clear, it is really simple to get things out, but one must be mindful of the fact that if a file is imported, then – as a general rule – any changes to that file’s name etc, should happen inside DTPO.

You can change it externally, but since DTPO isn’t ‘watching’ to see what you do to that PDF when you open it in an external reader, if you go and make filename-data changes to it, that make the change on the actual file sitting inside the DTPO db, then that file can become ‘orphaned’, since DTPO hasn’t been aware of the changes you make, and so cannot update its index to know what that file is now named.

Note, the fixing of an orphaned file is relatively straightforward, but the best practice is simply not to change filenames etc., unless you do it inside DTPO. That said, you obviously can annotate/re-order/change/delete pages inside the file, using an external application, as long as the filename isn’t being effected.

The ‘fix’ to the above, which is probably what would most interest you, is to simply INDEX files, rather than Importing them.

Files that are Indexed by DTPO, do not ‘reside’ inside a DTPO db. They remain where they are, inside your Finder structure. DTPO is basically watching the Folder (so one must be a bit circumspect about not moving/changing the Folder itself - but this can be cured by having DTPO index the topmost/apex folder, leaving you to move things freely below that uppermost folder), and as long as this Folder can be ‘seen’, you are free to make any changes you wish to any files/sub-folders below that.

[Ok. Maybe that’s a bit of an oversimplification - but the main point stands. :sunglasses:
One needs to experiment a bit in the beginning, to be sure one understands how DTPO and Indexed files operate – but it’s a fun way to learn things, and doesn’t take long to grasp, if you do rather than read !]

So you can delete/rename/copy/annotate/merge/add/consolidate etc. etc. to your heart’s content, as you would normally do, inside Finder – but having DTPO watching those files/folders, gives you all the power of DTPO in terms of search functionality, and its AI, along with - subject to some caveats - moving/splitting/dicing up the data inside DTPO.

I think its fair to say that most users probably have a combination between Imported and Indexed data inside DTPO, as opposed to only Indexing. Many (I would presume), probably started out there DBs with a complete Import, and then added more Indexed folders to the mix.

But the point of the above, is that the differences between an Imported library, and an Indexed library, whilst subtle, is significant, and might explain why you were running into the stickiness of trying to manipulate Imported files, as if they were Indexed.

This reply was far lengthier than anticipated. Apologies. But maybe something in the above will help!

5 Likes

Thanks for that explanation Brad! It’s very helpful. I’m still skeptical, but willing to give it another try with my articles to see if I can better understand these options.

Yes, that’s what bugs me most about it as well.

This. Do it. It dumps in your universal DT Inbox. Then you can bulk sort things later.

2 Likes

Correct. You should never put your DEVONthink databases in a cloud-synced location. DEVONthink specifically disallows database creation in such a location. It also will not open a database it detects is in such a location. It will warn you and give you the option of manually moving it or letting DEVONthink move it to a ~/Databases folder.

Okay, this seems like one of the most active Devonthink threads. How do you get academic citations from PDFs in Devonthink? I saw a screenshot in the past few days on here where someone showed an article with an academic citation. How was this achieved? I see there’s an option to import citations from Bookends.

In DEVONThink, I have a link (alias, whatever they call them) to my PDFs stored in Bookends (they are in iCloud). I do citations from Bookends. DT can do its thing (indexing, searching), and Bookends and do its thing too.

2 Likes

Actually, going forward, it would probably be more helpful to start new threads so others can read more focused discussions about DEVONthink.

4 Likes

Why would you advocate this?