Au revoir Bookends, bonjour Zotero

Thanks for the detailed reply @toffy. I‘m on the road today, so just quickly: Citavi’s backup file should indeed be the same as the Citavi XML file I mentioned. Given your requirements, trying to parse & import that file strikes me as the most promising approach. However, as mentioned, this would require some scripting.

The Zotero page that you linked mentions that, before exporting from Citavi, you could bake your PDF-specific notes into the PDF. That approach will likely also work for Bookends.

Also, since there’s already a supported import process for Zotero, it‘s worth trying that route first.
Additionally, if you send me a representative Citavi backup file (containing a subset of your refs & notes), I could have a look. I‘ll promise not to procrastinate! :-]

Thank you for that! Honestly, I’m not sure lawyers have those needs…or at least we aren’t set up for that type of functionality. We primarily cite to case law, statutes, and various secondary sources (mostly law review articles and treatises). There is one official public statute source per State. Most courts self-publish cases and then you have the century-plus Westlaw and LexisNexis duopoly on “case reporters.” Law reviews are inherently limited by the number of law schools in the US.

And that’s the other thing…American lawyers basically cite only to American law, so there’s no complication of foreign sources.

BUT that being said, at least the first 3 bullet points apply equally to lawyers. I think where you begin to lose me is the 4th bullet point and below. There simply isn’t “meta information” available for cases. And even worse, every court has slight variances in the so-called local rules for citations, so there’s not even a single standard to rely upon for citation style! If you are outside my home state of Washington, for example, you cite to our official Supreme Court reporter as follows: XX Wash. 2d YYY (Year). But if you are in Washington courts, that Wash. 2d converts to Wn.2d because…some judge said so at some point.

So maybe this is why lawyers don’t use citation managers. They don’t really help us!

1 Like

Currently, it’s definitely me who’s procrastinating… But at least it’s what they call “productive procrastination” as I need to get this reference management thing solved before I can properly transition to Mac.

Citavi → Zotero

Anyway, I have experimented with the Zotero route now and I’m no longer so optimistic about that route. There is quite some fixing that needs to be done, the biggest one being that relations between titles are currently and everything I find about that is that people say that this is how it is. So I assume there is a reason why apparently no Zotero translator can import relations between references. UPDATE: it turns out relations are supposed to be preserved but currently broken. See here.

I also found at least one item that was imported incorrectly and as a result lost quite some information. It is a book published online and I’m pretty sure I stored it as a Book in Citavi but Zotero imported it as a Webpage, which, in turn, entailed (at least that’s my guess) that the attached pdf was not imported. While this individual case can probably be solved, the problem is that there are probably quite some more similar edge cases that I may or may not discover before the final import, which means I may loose valuable data (I have been building my Citavi database over the past 15 years).

Zotero → Bookends

I did go ahead and exported a couple of references from Zotero to RIS (including notes) and imported these into Bookends. (I’m not sure why the bookends website recommends RIS for Zotero → Bookends, but I just stuck to those instructions.) Now, the first thing I noticed was that there seems to be a bug in the import script that cuts off the last character from each note/annotation. This turns out to be the closing > for the last HTML tag.

image

I suspect that it is due to these incomplete tags that the notes themselves are not rendered properly (they show as raw html), but when I fix the tags, it still doesn’t work, so the conversion, if any, must be done by the import script.

Also, I noticed that notes are imported into one single note and I suspect (hope) that this is also due to the faulty html tags, because there is no need for this, given that bookends also supports individual notes.

Citavi → Bookends?

Anyway, long story short: after having played a bit with bookends, I believe not only that it is far superior to Zotero in terms of features but also that importing to it directly from Citavi is probably the way to go.

One clear advantage of this route is that both Citavi and Bookends support notes on relationships but Zotero doesn’t, so this information would be lost on the way. Zotero also loses the direction of the relationship (which title is referring to which) and since there are no comments on relations, I see no way of fixing that in Zotero.

In Bookends, I haven’t been able to ascertain whether the links are symmetrical or directional but even if they are symmetrical, the comments filed could be used to indicate the direction.

There is one slight downside with Bookends compared to Zotero, though, and that is the absence of standalone notes. But that could possible be fixed by using one or more dummy references that can hold those notes.

One problem with both Bookends and Zotero is that annotations cannot be members of categories (Zotero) or groups (Bookends). This is a problem, because all my most important annotations are linked to one or more categories, indicating a specific paper or section thereof where I intend to use them. So I’ll probably have to write those categories as tags into the note itself.

So I’d be curious to learn more about what kind of work would be necessary to import from a citavi XML file. To get an idea of the data structure to be imported, you can look at this publicly available sample file at https://gist.github.com/zuphilip/76ce89ebbdac0386507b36cff3fd499a

Since you have so much invested in your knowledge in Citavi, something you might consider is using VMware to run Windows in Unity mode, then Citavi would be available to you while running Mac software. Whether this is useful probably depends on how integrated you want the reference manager to be with other Mac software.

From what I understood, Windows virtualisation doesn’t work with the new M1 processor… But I did start to contemplate using Citavi web, which is currently in public beta. Two problems with that, however:

  1. Citavi web has been in development for about 10 years now and while it is good to see it reaching Beta, something is telling me that we won’t see a final release any time soon.

  2. Even if it is released soon, I’m having difficulties seeing how exactly I would be using it on Mac. Obviously through the browser, but having to go through the browser will severely limit what I can do with it, right? They have a Word plugin that can talk to their API, but that may already be it. If the API will be public, things would look quite differently, though… But also in that scenario, time will be a heavy factor. I need something now…

Wow, I didn’t think I would end up right there before I even started using Bookends. I can’t believe just how non-caring Sonny Software actually is.

Just to quickly summarize my conversation with the developer, because it was via email, not on the “support”-forum. (Sorry, it became less of a summary than an annotated version of the exchange…)

I reported a bug related to importing pdfs from the watched folder:

I just tried the "watch folder” feature and suddenly the three references that I had manually added to my empty bookends database were done and it looked like this:

This looks like a bug to me but maybe I’m doing something wrong?

He replies:

I’m sorry, it’s not clear what you did. You had 3 references in the library and then did what? And can you reproduce whatever it was that resulted in this?

I confirm that I can reproduce the issue:

Yes, I can reproduce this:

  1. I’m using the demo version of bookends.
  2. I have three references in my database
  3. I add one or more files into the Watched folder
  4. I select “import from watch folder → start watching” (if it wasn’t already running)
  5. Once the import starts, an error message appears telling me that I cannot import more references.(one error message per imported file)
  6. The list of references looks like in the screenshot in my first email.

Does that help?

Te developer asked a few more questions:

Interesting. There’s no reason for that error message if you have only 3 messages.

Can you send me the following, please?

  1. A screen snap of the error message.
  2. The library file.
  3. One of the PDFs.

Also, where is your watch folder located? The Desktop? I wonder if this could be due to some permissions problem that could be fixed by providing Full Disk Access in System Preferences Privacy.

He also noted that

According to the screen shot you sent, you have 2628 references in your library, not 3. What is going on?

At the time, I read this as he scratching his head asking himself “What is going on” but in retrospect, he probably wanted me to explain what was going on. But instead, I tried to clarify the issue. (I have a lot of experience with support, so I know you can never explain your issue too much):

I gave you all I know. It’s the demo version, I added three refs manually and then I tried to add some more via the watched folder. It is very weird indeed:

  • Why is bookends adding all those messages as references?
  • How van it add more than a thousand references when there is a limit of 50?
  • Why does it suddenly start complaining at 2628?

Hope you can figure it out.

Will send you the requested files as soon as I can. But, yes, my watched folder is a folder on the desktop.

Yes, I also know that you always have to send whatever support requests, but I was away from the Mac, so I had to briefly postpone that. I am also used to support not really doing anything with the stuff you sent, but usually they somehow try to pretend (“I’ve forwarded your files to our developers”). Not so in this case:

Sorry, I have no idea. Since I can’t reproduce this, and no one else is reporting a similar problem, it’s a local issue and it’s unlikely I’ll be able to help. In this case, I suggest that you explore the other reference managers for macOS.

In fact, within less than 24 hours, he didn’t even seem much interested any more in the files he had requested. I sent them anyway, assuming that they would allow him to reproduce the issue or at least trace how it happened:

attached are the requested files:

  • the database
  • one of the files I was trying to import
  • and here is a screenshot of the error message:
    PastedGraphic-1

So the moment he has everything he wanted, he switches into PEBKAC-mode. Of course, the user is too stupid to realize that he exceeded the number of references allowed in the demo version:

this is simple. Your database has 2433 references!

PastedGraphic-1

You cannot add any more without a license.

He added the following:

P.S. I suggest that you trash this library file and start again. Create a fresh new library and try importing 3 references.

But in order to bring us back on track (i.e. bug-tracking), I chose to further clarify the problem:

Obviously, the number of references is the reason for the error message. But the actual bug is that

  1. I didn’t add those references
  2. They are not references but some kind of error messages or log-entries.

But to him the P.S. was very important:

You got my second email?

So I acknowledge that I received it while getting right back to the actual issue, this time breaking it down into smaller pieces and mentioning only one of them (first rule when communicating with evasive others: one - and only one - question at time! Don’t let them pick and choose which question to reply to!):

You mean the one where you suggest I should trash this library file and start again? Yes, this seems obvious to me since I cannot do anything with the current one except for remove 50 “references” at a time.

So, where you able to figure out what caused the library to be filled up with hundreds of strange items?

And indeed: I managed to make him answer the question, but he also pulled out a good defense: answer the question only in it’s literal sense and ignore any implicit meaning that would be obvious in a normal conversation:

I have no idea. No one has ever reported that before.

Well, strictly speaking, he wasn’t even answering the question in its literal sense, probably because that would have required him to say “No, I wasn’t able to figure it out, In fact, I haven’t even tried.” But even so, his “couldn’t care less” attitude is becoming quite apparent.

So I’m giving it one last try, by adding some humour and focusing on an a cue that he himself previously brought up (that this might be related to the location of the watched folder):

I guess if someone had reported it before, you’d already have fixed it and we wouldn’t be mailing about it… But hey, someone’s got to start…

Were you able to confirm whether the issue has something to do with the location of the watched folder as you mentioned earlier?

Now the blame is on “the system” (and ignorance):

The “imported” items are all code modules in the Bookends.app bundle. Why the system notified Bookends they were files in the watch folder I don’t know.

I advised you before to look at other reference managers, and I reiterate it now. We’re not going to figure out what went wrong in your case and unless it happens again there’s nothing to pursue. Please don’t invest any more time in Bookends.

Note the paternalistic tone and note the “we”. I would say I have done my part of the “we” figuring stuff out. If this is artisanal software development, it’s the kind that combines the bad sides of “artisanal” with the bad sides of “software development” instead of the other way around. The artist claiming unlimited authority over the art, not willing to treat it as a product that he actually has to accept responsibility for.

While writing this post, I realized that my bug is quite similar to the one @JohnAtl linked to above. So it is not even true that “No one has ever reported that before.” But that clearly is irrelevant to him.

4 Likes

Not to wag my finger as personal admonishment but … please appreciate that some will see airing of extensive complaints about software as being disrespectful.


JJW

1 Like

I’m not sure I appreciate it, but I’m aware of the phenomenon. I still haven’t quite understood how people can be offended by criticism not against themselves but against the software they use, not against the software they made, but the software they use.

But given that this phenomenon exists, I’m glad my post above is not about software but about so called support, and that it is not so much complaining but describing and analysing. If anyone feels I failed to mention one nevertheless important aspect of my exchange with the Bookends developer, I’m sorry about that. I should of course have said that the developer was extremely fast to reply.

3 Likes

I see it more as reporting than extensive complaints.
I understand @DrJJWMac 's reasoning below. Just an overview would be better.

Just saw this post - and thanks to you- now I will have to start reading axle.design daily- I blame you fully :slight_smile: Quite an interesting blog

Ha. Good thing there is nowhere near a daily frequency of posts. :sweat_smile:

Glad to be of service!

I’ll amend this way:

Have you posted your detailed description and analysis on the SonnySoftware site to see if any of the users can reproduce your problem and perhaps offer a solution?

In doing so, would you post verbatim what you posted here as describing and analyzing? Or would you amend it? If the latter case, why? How does any step to amend it make what you posted here really "not so much complaining?

The shorter form of your report needed only to say that you reported a bug, the bug appeared to be the same as the one from @JohnAtl , your report was essentially ignored, and you too are frustrated. The offense that I take is to the longer report with side comments on the details that you give behind the developer’s back (so to speak).

In a nut shell: I will defend your right to speak against software. I will however question anyone seeming to demean a developer, even tangentially, in a forum where the developer may not correspondingly be made aware in order to exercise their right to reply openly, even though I may feel the same way as you about that developer.


JJW

5 Likes

I usually prefer public support tickets/ forums but in this case, I followed the path of least resistance from within Bookends, which led me to write an email. Regarding reproducing the problem, the developer was only interested in reproduction until I said that I was able to reproduce it and provided the steps:

If the developer had suggested that I should check the forum to see if anyone else can reproduce it, I might have done so. But why invest time when the developer is not interested?

I’m not sure I understand you here, but since I didn’t post to the forum, I suppose it is irrelevant.

I’m glad you raise that point so explicitly because it allows me to explain why I disagree that such a shorter form of my report would have been better or more appropriate.

First off: I agree that shorter (as in more concise) texts have advantages. If you can convey something with X words, don’t use X + Y words. I take this principle quite serious. But there is another principle that is equally if not more important:

Let others speak for themselves as much as possible, especially when you disagree with them.

If I had written the kind of summary you are suggesting, I would have provided a summary of what happened based on my own relevance criteria, in my own words, and with an emphasis on my feelings. This is the kind of reports you find all over the internet and they are pretty much useless, IMHO, because they are more about the author than about the object they are reporting about. Such reports only become somewhat useful in larger quantities, like star ratings on certain online stores. If may people give a certain rating, that gives you some indication about the product. But one x-star rating?

My intention with my report above was not to add yet another complaint (1-star rating) to the pile of complaints about Bookends already out there. It wasn’t even primarily about the Bookends app itself. It was about the company behind it (which I suppose is the developer himself, but if it had been a huge software company, it would have been about the what kind of support the company provides and how it interacts with its users).

If my interaction with Bookends support had been publicly accessible on the Bookends forum, I would have done the same as @JohnAtl above and simply linked to it. Since it wasn’t, I had to provide it, not only in order to let people judge for themselves but also to be fair to the developer.

When I started the post, I intended to provide more of a summary, but as I went through the interaction, I actually found very little that I could cut out or rephrase without generating a bias. So I opted to provide the original text so that whoever cared to read my post, would be able to check my commentary against what was actually said.

So when you say:

I reply: my comments about the developers behaviour could perhaps be seen as demeaning if I hadn’t provided the evidence in which those comments are grounded, i.e. if I had provided a shorter version of my report. To state something like “I reported a bug but the developer ignored it” can be quite offensive against the developer as he may well respond “What do you mean I ignored it? I looked into it and said X. How dare you distort our interaction like that to make me look bad!” So I agree that, it in that case, would be appropriate to give the developer the opportunity to provide their view of what happened. But since I chose not to provide such simplifications, there is not much to dispute.

Of course, my interpretations of the quoted text can be disputed, but anyone can do that and I’m more that happy to listen to alternative interpretations of what happened. Indeed, I consciously formulated some of the comments somewhat pointedly in order to provoke contradiction. It would be incredibly enlightening to be told that I completely misunderstood this or that or that someone else might have reacted differently to when the developer said X and thus the exchange might have taken a different direction.

2 Likes

Regard carefully please that my alternative view is that you are posting the gory details in order to give yourself permission to demean.

And rather than beating around the bush, I will be blunt: If you cannot post your entire message in the SonnySoftware forum verbatim where it rightfully belongs, primarily to allow Jon to respond as is his right but also to engage in the community of users for help as the respectable and respectful path, then your approach here could also be considered to be taking a coward’s way out of the issue.


JJW

1 Like

Oh, that’s an interesting way of putting it. So are you saying that when someone conducts themselves in a way that is suitable to demean themselves, it is my obligation to protect them from that?

In my ears, the phrase “give oneself the permission to demean” is somewhat awkward because demeaning someone is generally seen as something that you should not do, yet the phrase suggests that it might be permitted under certain circumstances. But perhaps this is precisely where our disagreement lies: you seem to think that it is never okay “to cause someone to become less respected” (and hence, giving oneself permission to do so is basically a pseudo-permission, not a legitimate justification), whereas I think that if someone conducts themselves in a way that is suitable to cause them to become less respected, then it can very well be justified to make this known. Journalists do that every day. I wouldn’t use the word “demean” in those cases, though, but “reporting”.

Also, I wouldn’t call the details I posted “gory”, as it seems to imply some kind of voyeuristic intent (and, in the present context, that they were unnecessary). Rather, if they need an adjective, I would perhaps say that they are “telling”, because they tell the reader quite a bit about how the interaction went; which was, as mentioned earlier, exactly my intention with providing them.

I have no problem whatsoever with copying and pasting my entire message to the SonnySoftware forum, if that is what you were trying to tell me all along. The reason I posted it here was because I was already engaged in a discussion about bookends here and it made sense as an update to this topic. What would have been the aim of starting a new topic on the SonnySoftware forum with this content? If I had done so, I would probably have agreed with you that it would seem like I’m

with the main purpose of complaining (given that my complain was already rejected by SonnySoftware).

Last point:

I’m not sure where those rights are derived from. If Jon cares to respond to everything that someone says about him, then he certainly has the right to respond in the place where the original claim was published (and I’d be happy to continue the conversation with him here), but I don’t see why everyone who has something to say about Bookends or SonnySoftware support to do so on the SonnySoftware forum. - Of course, if the author’s intention is to complain and get a response from SonnySoftware, then that forum would certainly the place to go, but my intention was not to complain but to report.

I don’t see what would warrant such an interpretation. The only thing I might be “afraid” of on the SonnySoftware forum could be the software on which the forum runs (phpBB), but that’s mainly because I like discourse so much (which is what this MPU runs on). I don’t even see what issue I might want to get myself out of. In my exchange with SonnySoftware, I was trying to get into an issue (and failed).

Let us leave at this to not derail the thread.


JJW

2 Likes

I’ve had a moment to return to ReadCube Papers. I have an exhaustive database in it from about a decade of effort. After using it for a few hours simply to align two of my databases between it and Bookends, I noticed something significant. The friction for me to do simple things in Bookends was glaring. Here are just three examples:

  • Try sorting a list in the main panel to reveal files that have attachments or not. → NOT POSSIBLE
  • Try selecting a set of references in the main panel and exporting them as a .bib file via a context menu. → NOT POSSIBLE (you have to select files, mark them, and the use a menu option to export the hit list).
  • Try figuring out whether the paperclip icon in Bookends means there IS a valid attachment or there once WAS an attachment (but it is not here and you will have to go find it manually … (in a second window somewhere else)).

I have reached the moment of enlightenment … Grasshopper, it is time for you to go.

The one downside is that I could do what I might call “light-weight” syncing in Bookends. I would not have to carry around my reference database from more than a decade just to work on a new journal article. I am starting a list of requests on the Papers support forum to push for this as well as other features as I run across them (e.g. exposing the text edit fields to the macOS kernel so that we can run transformations or service options on them).

Goodbye Bookends, hello ReadCube Papers.

FWIW, the Papers iPad app appears also to be nicely done.


JJW

4 Likes

I seem to remember having read about Papers being tied to certain publichers, but can’t find the reference now. Maybe that was before it was acquired by Readcube. Anyway: do you see any signs of possible bias towards certain publishers or databases in Papers?

1 Like

I’m not the one you asked, but I’ll chime in. You may be thinking of Mendeley — they’re owned by Elsevier.

I don’t know whether Readcube has similar ties to any publishers, but if they do, I’ve not heard about it.

1 Like

Readcube was originally created by Labtiva which is now part of Digital Science. Digital Science has been part of Macmillan Publishers Ltd which, in May 2015, merged with Springer Science+Business Media to form Springer Nature.

Papers Mekentosj was acquired by Springer in 2012 (see this short history of Papers), and was again bought by Readcube in 2016 after the merger between Springer S+B M & Macmillan Publishers.

W.r.t. a possible bias towards certain publishers or databases, I can only speak for Papers v3 (before it was bought by Readcube): Although being owned by Springer, I don’t think that there was any preference or bias towards certain publishers or databases.

To fetch metadata and auto-download publicly available PDFs, we mainly used Crossref, plus the metadata from publisher websites as well as PubMed/PubMed Central or other databases that the user had access to (like Elsevier Scopus etc). The entire metadata & file retrieval process was fairly complex where one event (e.g. a DOI being found) triggered a whole lot of further events – like getting & parsing the article webpage for that DOI, and extracting further metadata like a PMID/PMCID from it, then fetching even more metadata from PubMed, and downloading the actual PDF from PubMed Central, etc.

4 Likes