DevonThink: how to increase the serendipity?

Software
1

I think I have a good handle on how to organize information in DevonThink. My inboxes receive text and documents from my mobile and desktop browsing, from email, from RSS, and from the work that I do. I do some processing with smart rules. I’ve a sensible array of databases with groups inside them. I have a taxonomy of tags. I have a nice thicket of wiki-linked articles where appropriate. I seem to not be losing things when I search to pull them back up. Etc.

What I’m almost completely missing is DevonThink contributing to my thinking, making surprising connections between items, suggesting filing that makes me see differently, performing word analysis on a document that increases my understanding, or anything else. DevonThink is succeeding at preventing me from losing and forgetting things, but I’d like it to do more than that.

Are there things I can do to get DevonThink to add to my knowledge beyond what I’ve contributed to it, or am I misguided in trying to make it do that for me?

I couldn’t find a question quite like what I was asking, but if we or the DT forum have covered this, happy to read existing discussion.

1 Like
2

How serendipitous—I am currently working on an upcoming talk on data and serendipity.

I haven’t refined my thinking yet (hence the work). Still, here’s the gist of it: how you conceptualize data before capturing it affects how usable that data is afterwards (or, more precisely, what you’re able to use it for).

In terms of records in DEVONthink, you apply your conceptualization of the data whenever you e.g., organize it in folders or query it with searches. To get creative, you have to take a step back and think about how to use those features to find new insights inside it.

Consider that data in any information system represents the real world. What uncertainties do you have about the real world represented in your DEVONthink databases? How might you explore that world?

Practically speaking, this depends on what’s in your database, and what kinds of questions you’re interested in answering. E.g., you might wonder if there’s some pattern to the things you’ve collected over time, so perhaps some date-based searches or smart folders will show you interesting trends.

The shape of the data isn’t the whole story, of course. There’s a lot you can do about functional fixedness/einstellung, cognitive overload, leveraging system 1 and 2 thinking… it all depends on what you want to accomplish.

One nitpicky gotcha: the tool can’t really make these kinds of connections for you. In my opinion, no tool can. If it were programmable, it wouldn’t be abductive induction. I think it will be a long long time before machine learning can emulate that human ability.

5 Likes
3

What did you read that created the expectation that DEVONthink would look at your data and add to your knowledge? Their advertising is somewhat misleading:

DEVONthink Help
The See Also & Classify inspector is the main interface to DEVONthink’s brain, our built-in AI engine. This engine is analyzing the contents and locations of all documents in your database and making connections between them. These connections can be seen in this inspector pane.

I’ve always objected to their use of “AI” – it is only a marketing term. Not “real” AI.

What DEVONthink actually does is look at a corpus of documents, extract a concordance of words, keep a running calculus of word frequency, look at the links you (not the software) created, the tags you (not the software) added, etc. It then simply makes mechanical suggestions based on word frequency and the other factors – basically telling your in the “See Also” display that “this document is ‘similar’ to these other documents”.

That is not knowledge. It’s a contextual analysis algorithm that is similar to “red things look like cherries”. DEVONthink can mechanically suggest relationships, but the knowledge-building can only be done by the user. Serendipity happens when chance connections between words (the only thing DEVONthink has) spark an idea in you, and you pursue it further than the software can take it.

Years ago a DEVONthink employee, now deceased, Bill DeVille wrote extensively in their forum about his use of the DEVONthink AI. Bill once wrote this advice, that I think might be helpful in framing an understanding of what DEVONthink can do for you:

Bill DeVille 12/4/2007
See Also is probably “smarter” than you think. It’s forte is finding similarities of words and especially the contextual relationships among words in a collection of documents. No, See Also doesn’t look at Names or at the group locations of documents (although classification may help you, the human part of the interactive team, organize your own thought).

Let me give an example. Dogs are canines. So are wolves, foxes and coyotes. Suppose you are viewing an article about dogs, which doesn’t include the term “canine”. You invoke See Also and find that the list includes an article about wolves, even though the term “dog” doesn’t appear in that article about wolves. How did that happen? Somewhere in that database is a “bridge” document that includes the term “canine” as related both to dogs and to wolves. The greater the number of such “bridge” documents, or the greater the frequency with which the relationship is defined even in a single “bridge” document, the more likely See Also is to make such a connection. [Emphasis added]

Take that as a tip. You are trying to force a connection among documents by grouping them. The connection may be the concept of stoichiometry, but that term doesn’t exist in many of the documents in your collection. One way to enhance the behavior of See Also to make that connection would be to make sure there’s one or more documents in the collection that “bridge” the term “stoichiometry” to other terms or word patterns common to the concept. That bridge document might be a beautifully written overview of the field, or it might be a “nonsense” document that is basically a glossary of related terms, perhaps repeated for emphasis.

I still do organization, at least to some degree, of my database collections for my own benefit. I can’t create and hold in my mind tables of all the tens of millions of words in my database and also the patterns in which those words occur, like See Also. But my database isn’t trained as a chemist, or ecologist, or economist or whatever my interests may be. So I’m responsible for determining the pertinence of documents suggested by See Also. Some of those suggestions may be “dumb” while others are “brilliant” – it’s up to me to make the distinction. This is human/machine interaction, and I often find it very useful.

Sometimes I find it useful to follow a trail of See Also suggestions. Perhaps the first list of suggestions may not give me what I’m looking for; but selection of a document from that list and another invoking of See Also may lead me to discovery of a relationship I hadn’t thought of.

(DEVONthink in 2021 is very close to DEVONthink in 2007, as far as See Also is concerned. So the advice still stands.)

11 Likes
5

FYI This should pull all his posts.

Here is the article you quoted from: Is classification of documents into multiple folders useful? - #4 by Bill_DeVille - Scenarios - DEVONtechnologies Community

4 Likes
6

@StephenL – thank you. Good to see they are not gone entirely.

2 Likes
7

I didn’t think they “purged” … not like them at all. Fake news.

8

The transition of the old DEVONthink forum (phpBB) to the new DEVONthink forum (discourse) on 12 March 2019 made all old previously stored URIs to old discussions unresolvable.

When correct, the forum should have retained all discussions since 2004 (see blogpost)

(note that some posts [e.g., 1] even date back to April 2003 :face_with_monocle:, almost 17 years of discussions, nice!)

2 Likes
9

Powers of searching comes thru again!

10

I got limited use from “See Also,” because I tend to manually group like documents together. So I’d run a “See Also” on a single document and find that nearly all the results were to documents in the same group–which I already knew were related–and the remainder were useless.

However, the last time I tried “See Also” I first manually excluded the documents from the same group, and found some interesting results.

11

Thanks for all the thoughts so far. It sounds like I’m not missing a major feature (sad!) but also that I can be mining more corners of DT to spark thoughts than I am currently doing. Especially enjoyed that quote from Bill DeVille; I never knew him and I miss him!

1 Like
12

@Kourosh’s recent interview might be relevant here: Kourosh Dini on DEVONthink – The Informed Life

From the email newsletter about it:

In a recent podcast I did with Jorge Arango at the Informed Life, I discuss how I use DEVONthink in managing ideas so they can playfully argue with each other, and form into solid writing works that I enjoy sharing. It’s a bird’s eye view of how I work with DEVONthink.

3 Likes
13

I became much less anxious about my DT use once I came to that same realization. It’s also why I’m intrigued by these long-term, hands-on, zettelkasten-ish approaches to knowledge development that advocate an ongoing “conversation” with my notes instead of waiting around for an algorithm to enlighten me.

2 Likes
14

This article by Steven Johnson is quite old at this point, but represents a story of how DT provides serendipity for him. This was the workflow that brought me to DT and I find it works for me as he suggests.

5 Likes