I am searching for an app that will allow me to set up pattern searches and combine the pattern searches in various ways over documents at the Finder level. Think: collections of file searches combined to create joint and disjoint sets.
My specific need is to search in combined ways on a collection of documents that span 10+ years teaching a course. Consider that I have topics as Composition, Bonding, Crystalline, … and categories as Exams, Homework, Study Guides, Demos, … The document types include PDF, LaTeX, Maple, Igor Pro, …
My example use case is as follows:
Search over all documents for contains:“composition” → keep this as a bin under Topics
Search over all documents for types: Maple → keep this as a bin under Types
Search within Topics:Composition for categories:exams → subset this under Topics:composition
In the end, my database collection should be akin to this …
Composition
Composition: Exams
Composition: Demos
Composition: Homework
Composition: Study Guides
Composition: Lecture Notes
Bonding
Bonding: Exams
…
For further background, Devonthink is not the best tool, as much as I had hoped it would be. To create anything as above, you essentially have to duplicate every smart sub-search and change the criteria. What I want is a way to create master searches and combine them with AND, NOT, or OR. So, I may have a need to find all documents that contain “composition” and do not contain “bonding”.
Ultimately, I will be eliminating duplicates and culling the files to a core set in preparation to write a textbook.
Suggestions to tackle what I need would be appreciated.
Do you checked the Devonthink Help?
There is a section regarding the rules for search operators. You can use Wildcard and Boolean operators.
So if you want to search for “composition” but not “bonding”, you should be able to find that with “composition BUT NOT bonding”.
I recognize already that I can create one complex search pattern in any app mentioned to this point. But this is NOT what I want. If it was, I would stay in Devonthink and be done. I want an app that allows me to create joint and disjoint sets among pre-existing search actions without having to recreate the gory, full-blown detailed internals on each pre-existing search action each time to kludge together the joint or disjoint sets.
I have perhaps two dozen topics. I have perhaps a half dozen categories. Stuff overlaps. Is there an app allows me to build searches this way
→ ((Topic A NOT Topic B) AND Category 1)
where Topic A, Topic B, and Category 1 are (complex) pre-existing searches? I do not want to do this …
copy the steps from Topic A search
paste them into a new search
copy the steps from Topic B search
paste them into the search with NOT criteria (if such exists)
group the results appropriately prior to the next steps
copy the steps from Category 1 search
paste them into the search with an AND criteria (if this is at all possible)
In addition, if such an app exists, does it allow me to save these combined search patterns and does it keep them dynamic as I add files to the folders that are being searched?
What about Textexpander?
You can create snippets, where you place your search topics in pull down menus the same as the Search operators. So you can call the snippet with a shortcut into the search field, select your terms and operators, and run the search.
On this way you can stay in DT and get the search you want.
[majorfrustration]
I should hope not to have to muck around building “hard-coded” searches in some other app in order to kludge together what I want in Devonthink. I will simply hard code in Devonthink if this is what it takes.
[/majorfrustration]
Alternatively, I would be glad when you could propose a Textexpander or KM or AppleScript method that allows me to select pre-existing smart searches in Devonthink with AND, NOT, and OR search criteria without having to “hard-code” the pre-existing search terms in the tool.
Today I have ((composition NOT bonding) AND homework). Tomorrow (when I finally get that far), I will have ((electrical NOT optical) AND homework). I should not have to rebuild the search on homework again. I should only have to build the search on electrical and the search on optical. Then, I should be able to build my new search as ((electrical NOT optical) AND homework). Almost with exactly this syntax.
I see files as data residing in a database. As little as I remember, robust database apps generally provide tools not only to search on the data but also to combine pre-defined search actions in various ways to create new searches. I have the same experiences programming to view (small) sets of data in science and engineering. I can search for data from last year, search for data where T > 120, and then combine these two as ((LAST YEAR) AND (HIGH T)).
I will try HoudahSpot especially as I have the SetApp subscription.
If I may make a suggestion here, if I understand the problem correctly, it strikes me that it divides into two steps.
The first step is to attribute a given document into one or more classes. These classes can be represented as “labels” (to use Google parlance) or, more commonly, “tags”. A document that is an examination paper containing questions on composition and bonding should be given the labels “Exam”, “Composition” and “Bonding”.
The second step is to combine these sets as may be appropriate using standard set algebra (union, intersection etc.) to these sets.
DEVONthink has as good a document classifier as you’re likely to find or need. @MacSparky’s field guide outlines how to train it and the documentation will take you further.
Building Smart Folder queries based on the presence/ absence of labels/ tags is straightforward. Tags are only one form of metadata and queries can, of course, also use other metadata values like the document type, date created, author etc.
Well, you might think of it that way, but that’s not how it is. Files are basically unstructured data on disk, with a few pieces of metadata, like creation date, update date, etc. As a matter of fact, all databases I know of save the data in files on the file system.
What makes a database powerful is that you must design a schema to be able to answer the kind of questions you want answered.
So this is probably the approach you want to take, create an actual database with a custom schema to answer the questions you need answered. It will be hard for some off the shelf app to just work the way you want.
Sorry, but I don’t understand your problem!
If you just want to search for “homework AND composition NOT bonding” than just type this into the search field of DT, and you will get your results!
And the next Day you just type “homework AND electrical NOT optical” in the same search field, and you will again get the results.
It is just that simple!
You don’t have to do this with a combination of a number of smart searches that are combined and, as the guys at the Devonthink forum already told you, only is something, you want to do so!
You also don’t have to become angry, just because I want to help you with your problem.
IF you don’t want to type those easy searches into the search window, you can do this with a preset Textexpander Snippet, where you can combine your search terms, and the possible operators, and do the search with a short cut, and some selection from a pop-up-menue over and over again!
And in case, you are complaining that the search I recommend would not place your results in a smart group, you can easily move, duplicate, or replicate your results into any group you want, and work with the results.
AND IF you want to work with a preset smart group, like one who is filtering the global inbox for the word “homework”, you can of course use the search terms (“composite NOT bonding”) also within this smart group.
I want to search ((COMPOSITION NOT BONDING) AND EXAM) where each CAPITALIZED term is a pre-existing smart search.
COMPOSITION
→ contains “composition”
→ contains “weight fraction”
→ contains “…”
(and perhaps later some more terms that I happen to discover)
I want to be able to redefine what is in COMPOSITION or BONDING or … as I go through the file system.
I do not mind designing schema, ie “smart searches”. I do mind not having the power to combine smart searches directly as citizens in their own right to create joint or disjoint sets.
Maybe this will offer an approach. I think that results from smart searches can be automatically tagged without manual intervention. The categories (homework, exams, …) seem ripe for this tagging.
One other thought. You haven’t given much detail about what you’re trying to achieve in organising documents in the manner described, but if it is to explore their content in detail and establish relationships between and categorise the detail — may specific exam questions and specific topics at specific levels of education — you might want to look at Tinderbox.
