Thank you I shall look at both Zotero and bookends.
Best wishes, DR
Iāve really, really struggled with the choice of citation managers throughout my PhD. Iāve ended up going with Zotero, because I know my data isnāt locked in (FOSS, works offline without a login), and because it supports the particularly weird citation management in my field via .cls files.
However, the UX in Zotero is far from ideal and I find it really clunky to manage different collections and quickly find relevant documents. I previously used Papers 2, which had an excellent UX (although was limited in other ways). To take one example of this: when you assigned a colour to an item in papers 2, the background of the whole row changed colour. It made visually identifying particular items very easy - I used to have book reviews purple, for example. In Zotero, this is done through tags, where you can then assign colours to particular tags - but you just get a small coloured square at one end of the item in the list, which is almost invisible. Similarly, editing tags requires switching a pane and is very clunky.
I feel there is a real lack of a good software in this space, especially for Macs - Bookends seems to be the closest. I have dabbled with this, which I think is closest to the āoldā Papers in terms of a very committed developer and passionate community, but Iām not a huge fan of the interface and when I tried it, found certain things somewhat counter-intuitive. I havenāt yet tried really committing to it for a project and perhaps it would grow on me with time.
Too many of these kinds of reference managers have problems, in my opinion:
- Some try to monetise via ongoing subscriptions - when you are building a life-time collection of data and research materials, this not good at all (and often import/export is limited or incomplete), as you are then tied into paying a subscription indefinitely.
- Some rely on online components, such as for logging in - some friends discovered this to their cost when Mendeley went down a few hours before a major conference deadline and they couldnāt access their references/update their papers.
- Some, especially those owned by large publishing companies, are definitely profiling you and your data.
- Some are very expensive (EndNote).
- UX is often poor, at best, though this varies a lot.
- Many are lacking features which some or many users would need, or have other limitations.
In the latter two items I would include, for example:
- BibTeX export
- a good companion iPad app (or easy standards-based way of accessing PDFs from tablets - some apps store these in a companion database that require workarounds to āexportā and re āimportā them)
- accurate āfire and forgetā downloading of metadata when you drag PDFs to the app
- incomplete metadata fields (very important in some fields - I had to give up on Papers 2 when I published some computing conference papers, because not all required citation fields were shown in the app even though they still existed!)
- ease of editing metadata. Papers 2 was excellent about this, Papers 3 required switching panes completely and was much more clunky. Zotero is middling in this regard - it doesnāt obey standard keyboard shortcuts, and there are a lot of ānice to haveā UX features that were in Papers 2 eight years ago which it doesnāt have.
And much moreā¦ I did once write a list of features an ideal app would have, once I finish the PhD Iāll dig it out.
5 Likes
I am still struggling, primarily between Papers 3 and Bookends (with Mendeley thrown in solely to offer team access to the Windows users in my group).
I ultimately believe that the best approach at the moment is to decide what matters most in the entire envelope of actions classed under ācitation managementā. Do you need to annotate to make intelligent connections on ideas? Do you need to read to grow your knowledge base? Do you need to organize to report coherently on your discoveries? Do you need to share to collaborate with a team? Do you need to ā¦ ???
You may find that you can better deal with the annoyances when you can find an app that provides the best benefits on the things that you absolutely need to be robust, consistent, and intuitive.
ā
JJW
1 Like
the best approach at the moment is to decide what matters most in the entire envelope of actions classed under ācitation managementāā¦
You may find that you can better deal with the annoyances when you can find an app that provides the best benefits on the things that you absolutely need to be robust, consistent, and intuitive.
I think you are spot on here, @DrJJWMac. One challenge is that some of the best-in-class features are not always standards-compatible, so itās not easy to pick the best app for each task and integrate them into a cohesive workflow. For example, I have dabbled with LiquidText on iOS which is phenomenal for PDF annotation and bringing together ideas. But if you use this, you have to import PDFs into LiquidText - which is a manual process from your reference manager of choice.
Dr Jeff Taekman, who featured on MPU back on Episode 169 nearly 7 years ago, uses a range of tools and brings them together into an academic workflow - he wrote up his workflow in 2015 and recently updated it. It involves a lot of moving parts but one doesnāt necessarily need all of it, depending on oneās specific requirements.
Jeff uses BookEnds for example, so I really ought to go back and look at this when I give BookEnds a proper workout. However implementing a workflow like this is most definitely a distraction activity from my thesis write-up! Iām also avoiding upgrading from 10.14 on my 2014 rMBP in the middle of the thesis, so apps like LiquidText and the latest version of Highlights are not available to meā¦
Perhaps Jeff Taekman would be a good returning guest to have sometime for another episode @MacSparky? Caveat: Iāve not yet listened to the Research episode this week, so there might also be interesting ideas there!
2 Likes
@sunil - Youāll be pleased to know Jeffās already agreed to come back. I working on planning that episode now. 
3 Likes
Here is my story. I have been using paper since 2010 and I loved it. Even with Papers 3 I was ok given the limitations. I use it only to organize and read PDF peer review papers. For citation management I use EndNote since 1993. I have more than 4,500 pdfs and I am scared that with the Readcube, Papers3 is going away soon. In my use the ReadCube Papers is a very limited in a couple key points.
- not allowing to email PDF attachment
- allowing to short library by very few fields. I have more than 150 publications and I number all my paper based on the number that they appear on my CV. In Papers3 I use the number field and I add my number. Then I can short based on this field. In ReadCube it is not doable.
1 Like
Not sure what your specific requirements are with regards to your dissertation - but finally finished mine (thirteen years later) at the start of 2020.
I had about 3500 references inside Bookends, where I kept ācleanā versions of my sources. On importing into BE, DT3 would also be watching that same folder, and I would then have it duplicated into DT3.
In the latter, I did all my annotations, reading, most of my note-taking (shared with LiquidText, MarginNote and Goodreader on iOS), and linking, searching and sorting.
I absolutely hear you about being able to use any number of tools for much of the management of your sources, but noticed one glaring omission (unless I missed it?) ā and felt compelled to write a reply: the automated bibliography.
Do not discount the benefit of that, without serious consideration. My thesis is no doubt an outlier, given its length at over 1000 pages, with a reference list/bibliography that spanned 75-odd pages alone. Using BE (or any reference manager), meant that each time I compiled, that biography was compiled automagically ā which saved me countless hours in the end at my having to verify each and every footnote (almost 5000 in total), to confirm that whatever was mentioned there, was listed in the bibliography. Similarly, I reliably knew that was listed in the bibliography, definitely appeared somewhere in a footnote. This feature alone makes a reference manager worth it, in my view.
Lastly, if your dissertation is a once-off, then no worries. But if you intend to continue publishing, keep in my that reference styles could differ in various journals. In my field/my neck of the woods, just about every journal I could publish in, has a unique referencing style. With a citation manager, that requires me setting up a single style format, once, per journal I intend submitting to, and the rest is done.
All of the above is not new, or even news, but feel compelled to urge anyone who is starting out, to be very sure about their decision not to use a reference managerā¦ What might work fine now, could be far more painful down the road.
6 Likes
This appears interesting as a new kid on the block.
Maybe parts of my earlier dream of being a wealthy programmer are coming true for Christmas this year.
Sidebar: I have to wonder how this compares in what @msteffens noted for developments on Keypoints.
ā
JJW
1 Like
i havenāt tried Zenreader yet but it seems to share some important ideas and concepts with Keypoints, like focussing on knowledge snippets and integrating PDF annotation & bibliographic metadata support.
From what I can see, Zenreader mostly differs from Keypoints in that it is an (Electron-based?) crossplatform app. So itās also available on Windows, and they say Linux, mobile & web versions are ācoming soonā. Being on the Mac, I personally like to use native versions which offer native controls & behavior and which can be integrated well with operating system features (esp. AppleScript for scripting/automation & integration with other apps). OTOH, I definitely see the advantages of being available on other/most platforms from the start.
(I guess) both apps use a database in the background to support its various features. However, Keypoints is focussed on a plaintext/Markdown workflow, and puts its plaintext notes front and center, parsing their freeform content into a navigatable graph of knowledge elements.
Another big difference: Zenreader is available now while Keypoints isnāt even available for testing yet.
Can you expand on that part of your workflow a bit?
How do you automatically get those files from DT3, which acts like the storage bucket of your annotated PDFs if I understood correctly, into LiquidText/MarginNote/Goodreader?
I bought LiquidText and MarginNote and find them both quite clunky. LiquidText on the Mac is basically unusable due to performance issues with any document that has more pages than a short conference paper. And the lack of sync between LiquidText for iPadOS and macOS was their biggest mistake in my opinion.
Thanks in advance 
My workflow needs a real revamp!
1 Like
There is unfortunately not much to expand on. I was wrapping up my research and writing when DT3 came out, and so havenāt really have an opportunity to dig into what I could in terms of the new automating features. But not even clear if that would make much of a difference, since I was posting things to iOS, which was covered, in my case, suitably by DTTG.
In essence, I had a folder action set up to watch the BE folder ā when a new PDF was inserted there, I would then be prompted for the folder I wanted the PDF to go, as an imported item, inside my DT database.
Back inside DTPO2/DT3, I was fortunate in having enough storage available on my iPad Pro, and so I simply had my various research libraries/databases fully synch over to DTTG ā in my case, using the direct ābonjourā wifi sync. Once over in DTTG, I would then simply open those files in Liquidtext/ MarginNote / Goodreader ā depending on what it was, and what I needed ā annotate away, and save back.
That said, before I settled on the latest workflow, when still using the more clunky predecessor of DTTG, I simply replicated the files I wanted to annotate over to a specific group in DTPO2 ā and only synced that group over to DTTGv1. I would then āopen inā, annotate in the others, and save back. I would then sync again. Back in DTPO2, the PDFs in that folder, being replicants, had now transposed their annotations to the āoriginalā PDFs in their āoriginalā locations ā so I would then simply delete all the replicants that I had just synced back, leaving the originals, now annotated, where they were originally. I would then repeat the process with the next batch.
To be clear, all of the above seems to be far more complex in writing about it than what it actually was/is, both in DTTGv1 and the newer DTTG. All of the above took mere minutes per batch, and relative to the many hours spent reading and annotating over on the iPad, was honestly no hassle at all.
So in short, still a manual process ā but a really simple one. I therefore donāt have a fully-automated version of doing this yet ā but when I get back into the saddle, and look at things again ā if something gets put together that works, will be sure to share!
Ah okay, thanks a lot for the explanation!
The initial write up made it sound as if you were also making use of MarginNoteās and/or LiquidTextās features to collect and manage files and especially build a knowledge database across multiple files.
Thanks a lot for the tip to have it watch folders. Iāll have to dig into those sorting prompts.
1 Like
I am not sure whether it is best to resurrect this old thread, or start a new one, butā¦ I wondered what MPU folk were using for academic literature storage, referencing/citation and management these days. I realise that this is a complex discussion (you can go down rabbit holes for the best tool to e.g. highlight PDFs, surface/assemble scraps of knowledge, or whatever) but Iām interested in whatever!
I stuck with Zotero through my recently submitted PhD - the viva is in a few months, and I knew that testing out alternatives before that would be a very effective distraction activity!
For citations Zotero generally worked well - although Word did slow down notably as I merged the different chapter documents into a single 300 page thesis file with each chapter containing many citations. I held off doing this until the last few days before submission, which brought with it a few other challenges, but Iām glad I did so: including the TOCs and so on, my 2014 13" i7 MacBook Pro was taking a good 5-10 seconds to save the file on that last day.
However, Iām still very dissatisfied with the user experience and UI of organising/managing papers in Zotero, per my post in October 2020. Zotero has recently introduced a much improved internal PDF viewer, which (bugs aside) is very good and has a lot of potential - e.g. for collecting individual highlights/comments and tagging them - though it isnāt as flexible as apps such as LiquidText/MarginNote. I would also note that in the last year or so (I canāt remember exactly when) Zotero has introduced an iPad app which although still in beta, allows for viewing information about your references and viewing the PDFs themselves. I havenāt tested this out lately, but I assume it is also linked to work on the PDF viewer on the desktop app, so that highlights, annotations etc would carry across.
But I am still frustrated by relatively simple things:
- Colour-coded items (coloured tags) are barely visible in the list of references, unlike e.g. Papers of old. Managing/using Tags is also very clunky.
- Searching large libraries is frustrating, especially if the search term is a common word or stem. It isnāt possible, as it used to be in Papers 2 (and possibly 3) to narrow fields down in the search bar, e.g. searching for things where the author field contains Jones with āau:Jonesā
- When you import duplicates (again, very easy to do with a large library) there is no duplicate checking. You can manually view duplicates later, but merging duplicates is a manual process where you have to select the relevant items one at a time. It isnāt possible at present to flag false positives, which is also frustrating.
I do feel churlish complaining, as it is FOSS software and the basic functionality is very solid. If I had suitably advanced coding skills I would happily contribute to the project. Unfortunately I donāt, and - like many FOSS projects - the bit that seems to suffer from a relative lack of finesse is the user experience.
In relation to previous MPU content, I notice that Aleh Cherp (MPU 221, 2014) updated his list of āessential academic softwareā last summer. I may have missed it in the the stress of the final year of the PhD, but I donāt think the updated episode @Macsparky mentioned with Jeffery Taekman has yet come to fruition.
So overall I would be very interested in hearing other peopleās perspectives on their current workflows. Iāve bought a license for Bookends in the recent Winterfest sale, so will be giving this a good shot too.
Iāll be honest and say that I have only really used Zotero and not the alternatives. Itās been an amazing tool for academic writing and reference management. CMD-SHIFT-F gives advanced search which lets you search for e.g. creator or year. Not as fast as au:Jones, but a compromise.
I use Zotero with on iPad using the new built in PDF editor to highlight and then sync with my Mac. It has worked well (although necessitated me buying space on my Zotero account despite having lots of iCloud space available). Iām still not 100% sold on the need for it, as opposed to integration of Zotfile functionality which allows files to be transferred to a tablet and highlights to be extracted. Was the Zotero PDF editor required when there are so many alternatives? Unfortunately Zotfile which gave complete flexibility over file management increasingly looks like abandonwear (Is this project dead? Last commit almost a year ago Ā· Issue #522 Ā· jlegewie/zotfile Ā· GitHub) with one minor compatibility fix in the last year.
I used to be an avid user of Papers3 and made the switch over to ReadCube when Papers3 support was dropped. It is by now means a perfect application but I am pretty happy with it overall.
I find that matching works well and I like being able to annotate the papers on my iPad. Depending on your work situation, I find that being able to access my library online away from my personal computer pretty helpful as well. There is ability to tag each reference with a different color, I think search isā¦okā¦but not perfect, and Iām not aware of a method to find duplicates. I hope you can find something that works for you!
1 Like
Bookends is absolutely the most powerful reference manager available. Nothing comes close. It has a bit of a steeper learning curve at first but itās worth it. Its only downside is that itās nowhere near as good as Zotero at getting metadata for books and older journal articles. If you only need to use PubMed, etc. that wonāt be a problem but anything published more than 15 years ago will probably have imperfect metadata. This limitation can however be overcome: How can I use Devonthink 3, PDF Expert and Zotero together? - #3 by prc
Also re: searching Zotero, I use Alfred and the zothero workflow, which does make it possible to search by fields (though, you have to select the field first, which is a lightly disruptive additional step).
All my PDFs are stored and synced across devices on Box.com, which thus preserves highlights but breaks Zoteroās ability to open files on mobile devices (but is fine on non-mobile).
A few months laterā¦ I had thought that submitting the thesis would make time for testing out new workflows, but instead my day job took over! I passed my viva last month and Iām finally trying to make time for testing things out.
@nationalinterest - thanks for the Cmd-Shift-F tip, that one had passed me by entirely. Yes, I noticed Zotfile seemed to have become abandoned. Itās one of the challenges of FOSS - a lot of functionality is delegated to third party plug-ins, which then increases oneās dependency on more than one product.
I think the closest in terms of Mac-like philosophy to Papers (v2 - v3 never grew on me) is BookEnds. Thanks for the warning about metadata @prc. One of the joys of being an interdisciplinary researcher is that I have a wide range of sources from different fields/time periods - it certainly isnāt all PubMed. But your link seems like a possible workaround.
@Lutefish - I hadnāt thought about using Alfred as a way in to my Zotero library. Interesting idea, especially as I use Alfred already, thank you!
Iāll have a play over the next few months and report back.
Incidentally the other day I was looking at Agenda (made by at least one of the people who was originally behind Papers, Alexander Griekspoor - and in particular some of the lovely user experience touches that are nicely demonstrated in the video outlining the latest release. It made me really miss some of the same kind of thoughtful user experience that Papers v2 had, and - as someone who works partly in human computer interaction (HCI) and UX - goes back to my primary frustration with Zotero (and many other āprofessionalā tools - looking at you, nVivo and SPSS!) which is that the UX is at best clunky. Plus Ƨa change!
1 Like
I use Bookends, it is a good app, nothing does everything of course. I only use it to have formatting at hand so to speak and I hardly ever use it. In fact I have gone back to notebooks for keeping references.
Otherwise frankly I am thinking of going back to LaTeX biblios on my Mac: I find it sufficient and I am getting a yen for the simplicity it offers (offered). As you say there is a kind of skill attached to it but, you have everything under your hand in some sense?
I do have issues with the citation mania, publish or perish culture and much else that would take me outside the brief of this site: however they are connected.
I think we are in danger of getting over-ITed if there is such a word. For large numbers of folk the real issues are with access and so on not the IT end as it were. There is an invisible battle by some academic publishers to keep their paywalls intact, which is getting harder and harder for them and might well be impacting the behavior of some bibliographical apps, though I understand little of the technical side. Thankfully in my view. I appreciate there are legal issues.
I also find in my own case that less and less of my concerns are with the IT I use, which is now frictionless to be fair to the Apple platform and my beloved apps, and more with āsocialā aspects, copyright issues the overproduction of papers on many topics etc. etcā¦