I’ve got a filesystem that looks like Documents/Finances/Bills/Water/2021-02-05_water-bill.pdf. I’d like to convert that to Documents/PDF/2021-02-05_water-bill.pdf with the finder tags of “Finances”, “Bills”, and “Water”. This seems like something a computer could do. I’ve got Hazel and Keyboard Maestro, has anyone else used those tools to do anything similar?
Hazel can certainly perform the required actions, but I’d recommend you make a copy of the hierarchy (or part thereof) for testing first.
Among Hazels actions are “Add tags” and “Move” … “To enclosing folder”.
At the simplest, you could create a rule for each directory but that might be almost as much work as doing it manually. There may be a way of using the special “match” rules to use the path elements as the tags, but I’ve only just managed to get my first match rule (almost) working so will defer to more experience automations experts here.
Another option would be some scripting but again I will defer to others.
I think you are looking at a project that is going to be a bit more work than you might think at first, depending on how many file you have and how many folders are involved, and whether the folder depths vary (eg you have Documents/Finances/Bills/Water/, but do you also have Documents/Finances/BankStatements/Bank1/Account1/ (note that the second path has one more level of subfolder).
While you could do this with Hazel or with KM, I think that you would need to create so many different rules that the effort would not pay off at the end, because this is a one-off task; once you have reorganized your files you presumably won’t do this again.
If you have only a small number of folders, you might want to do this “by hand.” You could tag one file with “finances” “bills” “water”, thus creating the tags, then select all the files in that folder and drag them in Finder onto those tags on the sidebar, which will assign the tag to all dragged files, then move the files to the required final location. Obviously if you have more than a few folders to do this to, which I suspect you do, this is a time consuming and error-prone process.
The way I would do it is by writing a custom script for the purpose, recognizing there will be a significant effort involved, but the reliability of the process once the script is in place would make that worthwhile.
Let’s assume the path to every file is of the format Documents//filename, and the idea is to use EACH of those as a tag.
I would write a script that took a file path, parsed out each of the intervening subfolders, tagged the full path with those names, and moved the file to the target location. You could wrap that code in a loop that recursively descended through folders, or recurse via other tools like ‘find’.
For example, in Python, this code would probably do the trick (note that I haven’t testing this other than trying out the syntax in the Python 3 interpreter). Also note that this script moves the files to /Users/me/NewPlace, which is NOT under the Documents folder. This is because it’s a pain to exclude a folder when using the find utility and it’s just easier for this purpose to move all files somewhere else and then relocate the NewFolder into Documents afterward. Also, this script DOES not delete the old folder structure (but the files themselves will not be there as they are being moved).
You could modify the script to COPY the files instead of moving them, or to remove folders once empty as well.
#!/usr/bin/python3
import os.path
import os
import sys
# Assume that all filenames given are FULL paths, eg /User/me/Documents/A/B/C.../filename.ext
# Assume we are putting all files in /User/me/Documents/NewPlace
# This is the part of each filepath that we will NOT include in the tags.
# Note the trailing slash is important.
ignorePart = '/Users/me/Documents/'
targetLocation = '/Users/me/NewPlaceForFiles'
# Loop through all the filenames given
for thisPath in sys.argv[1:]:
tags = thisPath.replace(ignorePart, '').split('/')[:-1]
SetTagsOnFile(tags, filepath)
filename = os.path.split(thisPath)[1]
newPath = os.path.join(targetLocation, filename)
os.rename(thisPath, newPath). # move the file
Assuming this script is saved on your desktop as a file called ‘utility’ (and has been made executable), you could process your folder tree under Documents with something like:
find /Users/<you>/Documents -type f -exec /Users/<you>/Desktop/utility {} \;
The -type f part restricts the find command to finding only “regular” files, eg it ignores the folders themselves, and the -exec part runs the utility, with {} inserting each filename. The slash before the semicolon is important to the shell syntax.
The rub is the SetTagsOnFile part, because there is actually no easy way to do this in a scripting language like Python. AppleScript sadly does not support setting Finder tags either, which just shows how Apple is not really building AppleScript any further.
There are a copy of ways you can handle this. One is to have Python execute the appropriate shell commands to assign the tags, but this winds up being a bit complicated with the need to execute a subprocess that gets the existing tags on the files, builds a new tag list, and then sets the tags on the file, and since tags are actually stored in the file’s metadata as a binary formatted plist, the whole process is rather a pain. If you want to go down that rabbit hole, I can give you my code for accomplishing all of this, but if you are not versed in Python and the shell commands you are going to find it a bit hard going.
Another option is the “tags” utility which is freely available, and I know the source code is on GitHub, and that seems a popular utility.
I have also written my own version which I call Tagger, which is written in Swift, and I am happy to provide that (without warranty) to anyone who wants it. I use it myself extensively called by python and shell scripts for this purpose.
Sadly, this has gotten rather complex for what you would think would be a relatively simply process, but the idea of converting a variable folder structure to tags is not something that seems a common enough task for someone to have built it into a pre-made utility.
I suspect you could build something like this in KM as well, and in the end that might wind up being easier. I would do it in code because I have already built a lot of the tagging infrastructure for a number of utilities that I wrote for my own purposes.
If you want to pursue this approach, I can give you more help. If you have KM, I would consider playing with that a bit. As already noted, you can add tags to a file from KM, and I know you can store the names of tags in a variable (comma separated), and use that variable to set the tags on a file as well, so that is a viable solution as well.
As @zkarj says, create a duplicate of your folder structure and use that as the source of the files to do this to in case things go tragically wrong.
3 Likes
Turns out that KM has an action to loop through every file in a Folder with an option to do this recursively. Use that action to get the full file path for every such file. You could embed the python script I posted to generate the necessary tags, or write that string parsing in KM (that is a bit harder; KM is not necessarily built for that) and use KM to set the tags.
I would suggest using KM with an embedded script to parse the path to get the needed tags and use KM for the loop and setting the tags. That will be a much simpler solution that what I proposed above and is better suited for a one-off, unless you want to go down the rabbit hold of coding to create the tags.
(I should note that if you happen to be familiar with programming in Swift, it’s quite easy to do the tagging part with a few lines of code in Swift. I am finishing up a utility to tag files with their related year and month (as was mentioned in the latest MPU episode) because I would like to routinely do that going forward; I intend to have Hazel use the script to automatically add these tags for me and will also invoke the utility manually (through a KM trigger) as needed as well.)
2 Likes
Thanks for pointing me in the right direction, this script got the job done, along with a command line tool called tag https://github.com/jdberry/tag:
#!/usr/bin/python3
import os.path
import os
import sys
# Assume that all filenames given are FULL paths, eg /User/me/Documents/A/B/C.../filename.ext
# Assume we are putting all files in /User/me/Documents/NewPlace
# This is the part of each filepath that we will NOT include in the tags.
# Note the trailing slash is important.
ignorePart = "/Users/me/Technical/"
targetLocation = "/Users/me/Technical2/"
# Loop through all the filenames given
for thisPath in sys.argv[1:]:
tags = thisPath.replace(ignorePart, "").split("/")[:-1]
tags.pop(0)
tags.append("Research")
tagString = ','.join(tags)
command = "/usr/local/bin/tag -a " + " \"" + tagString + "\" \"" + thisPath + "\""
print(command)
os.system(command)
filename = os.path.split(thisPath)[1]
newPath = os.path.join(targetLocation, filename)
os.rename(thisPath, newPath)
I ran the script with this find command, edited to weed out hidden or special files:
find . -type f -not -path '*/[@.]*' -exec /path/to/tagsystem.py {} \;
Worked well, I should mention though that this script does move the files, not copy them. You’ll wind up with an empty directory tree after running this.
1 Like
Glad you got this figured out - I was going to suggest GitHub - jdberry/tag: A command line tool to manipulate tags on Mac OS X files, and to query for files with those tags. and recursion in your script.
Glad you got it done.
Tag is the utility I was referring to. I wrote my own version because I wanted it to work a bit differently, but tag is a very nicely done utility.