Discovering the Power of Plain Text w/ MD and Links for book project

[Bmosbacker]

I recently created a quandry for myself. I have been working on a large book project. Although I’m making good progress, I have spent a bit too much time experimenting with two primary writing apps for this project: Ulysses and Scrivener. Without going into details, Scrivener is hands down the most powerful, feature rich program for large complex writing projects. This strength is also its downside. Getting text out of Scrivener is also a complex process using the Compiler. I’ve found that I’m only using 25% of the features in Scrivener.

Ulysses is less complex, uses plain text, but requires a subscription. I am diligent about keeping subscriptions to a minimum (no need to defend subscriptions–that has been beat to death).

Then Obsidian comes along. After I discovered the Kindle plug-in I realized I could scrape all of my research highlights and notes from well over 100 Kindle books related to my book project into a reference folder in Obsidian for quick referencing and linking within Obsidian. I also discovered the footnoting plug-in.

So, I exported my Scrivener text and imported everything into Obsidian. I realize that the following benefits will be obvious to long-time MD/plain text users but I’m listing them for those who have less experience.

• I can access my book text using nearly any app I choose–my text is absolutely platform/app agnostic and future proof.
• Everything is lightning fast, especially on my M1 MBP
• Backups are fast.
• I don’t have to sync with Dropbox (Scrivener requires Dropbox syncing)
• I don’t need the complexity of Scrivener. Obsidian plug-ins allow me to customize and limit features to only what I need.
• I’m getting more writing done in shorter time. This reminds me of my productivity when I used Byword.
• Linking ideas, research notes, etc., is fantastic in Obsidian and much easier and faster than in Scrivener.
• The graph may (I haven’t spent time with this yet) lead me to discover connections between concepts I’ve not seen before.

A few Caveats
There are a few things I lose in using Obsidian.

• Scrivener has a feature called Scrivenings. This feature enables one to select multiple files and see them in a contiguious flow of text–this is helpful in assessing the flow and transitions between sections and chapters of a long book. Does anyone know a way to do this?
• It is easier to insert and manage citations in Scrivener
• I have not found a way to export to or import into Word for final formatting. Scrivener’s compile feature is designed for this in addition to compiling for various outputs and formats. However, the good news is that if needed, I can import MD files into Scrivener for compiling if needed.

Does anyone know of a way to get multiple plain text MD files into a single Word or Pages doc?

• It is not possible to manually sort files. This is frustrating when I want to rearrange the order of chapters in the book.

I’m assuming I’ll continue using Obsidian but Obsidian aside, I will complete the book using MD plain text.

[ibuys]

@Bmosbacker

Does anyone know of a way to get multiple plain text MD files into a single Word or Pages doc?

Pandoc could probably do this.

[jcarucci]

I finally settled on using plain text for my notes after trying a bunch of proprietary formats, including Evernote and Apple Notes. I’ve been using iA Writer, but now experimenting with Obsidian. The best part about this is since iA Writer and Obsidian both use plain text, I don’t have to worry about how to get my notes out of Obsidian, if/when I decide to switch. The pain of getting notes out of Evernote has made me vow to never go down that path again. Plain text forever!

[tf2]

You can get something approaching Scrivener’s scrivenings feature in Obsidian using file transclusion. The syntax is, irrc:

![[transcluded note name]]

That said, as powerful as I find Obsidian to be, I’ll keep using Scrivener for long form writing. I. The spirit of Luhmann, I see a sharp distinction between drafts and final manuscripts on the one hand and notes on the other. Even if you could emulate most of Scrivener’s features in Obsidian — a good bet, tbh — it would take far more effort and fiddling.

[JohnAtl]

@Bmosbacker

Does anyone know of a way to get multiple plain text MD files into a single Word or Pages doc?

Here’s one way:
Open a terminal.
Type cat with a space after it.
Select files in finder, then drag and drop them into the terminal window.
(you’ll see all the filenames listed after cat )
Type | pandoc -o chapter.docx
(Note that the first character is the vertical bar, aka. shift+\)
Press Enter and in a few seconds you’ll have a Word file.

Prerequisites

Install homebrew https://brew.sh
Install pandoc brew install pandoc
(or install pandoc another way)

[anon85228692]

My own two cents: I don’t think it’s a great idea to use one or the other. I use both in fiction writing: Obsidian excels at ideation, while Scrivener excels at production. Free form linking in Obsidian allows you to build in the random directions that make the bulk of creation. Scrivener is unparalleled for getting a massive writing project done. I have both (+ Craft) open all the time, as ideas and scene outlines get fleshed out into a real first draft. And it’s reasonably easy to separate them as I find that building is quite a different step from the writing itself.

[RDK]

@Bmosbacker

sections and chapters of a long book. Does anyone know a way to do this?

You can preview several files at once in Marked 2. Feature called Multi-file Documents. Then you can copy result to the clipboard.

[Rob_Polding]

I’ve also moved everything into Obsidian, including my Ulysees documents. I’m really happy to have everything in one place. Now I’m only using Drafts to capure and Obsidian for all my reference and writing, as I have come to hate searching over multiple apps and love the customizability of Obsidian.

[Bmosbacker]

I don’t disagree, Scrivener is designed for large complex writing projects. But for whatever reason, I find I can write better and and produce more using a plain text editor than writing in Scrivener. I may well use Scrivener to compile and publish once the writing is done. Scrivener has an excellent focus mode but overall the layout is very busy. I’m finding that I prefer using iA Writer linked to Obsidian folders to get the text written.

[Bmosbacker]

This is very helpful, thanks!

[Bmosbacker]

@JohnAtl I appreciate this help but I have a phobia about using the terminal–something I’ve never done. I fear that “terminal” may indeed describe my experience if I try!

[drezha]

Thanks for posting the thread - I had no idea there was a footnotes plugin for Obsidian, so that’s been downloaded.

As others have said, Pandoc is probably the tool of choice for converting markdown to docx. However, I have in the past used both Marked 2 and Typora to do that. However, I’ve found pandoc to be the best option, especially as I can feed it a template .docx document and it’ll output the file based on that. Has made writing some long form university documents easier than writing the whole thing in Word. However, I do have to go back and tidy some items up, but it’s not the end of the world in comparison to writing it all in Word (it’s quicker to type ### than it is to select Heading 3 for example!)

[Bmosbacker]

@drezha I had high hopes for Pandoc until I read this, “Step 2: Open a terminal…” I’m afraid (literally!) that I don’t do terminal.

[anon85228692]

I may well use Scrivener to compile and publish once the writing is done.

Be aware that moving a writing project in Scrivener for revision and compilation is harder / longer than it seems.

[Bmosbacker]

Yes, when I try to import text files to Scrivener and then compile/export those text files from Scrivener to say Word, the MD syntax remains (there may be a way to eliminate this in the compile function but I’ve not discovered how), which creates extra work that would not be necessary if I wrote everything in Scrivener. However, iA Writer does an excellent job of exporting MD text files to Word format and strips all of the MD syntax. Here is an example from a draft section of my book:

iA Writer MD Text

Result in Word

I think/hope that once I figure out how to use MD Transclude (a new concept for me), I should be able to combine whole chapters in iA Writer and export to Word for final formatting and completing citations and references. I’m experimenting in small chunks to master the workflow to determine if in fact this is worth it instead of just using Scrivener. Thus far, I think this will work well.

[drezha]

Ah, that can be off putting, but it is one of the easiest command line software’s I’ve used.

This is a good guide I’ve used previously - though it is LaTeX to Word, but substitue Name.tex for Name.md and it’ll work.

[brookter]

Have you looked at Scrivener’s External Folder Sync feature?

You can write documents in another editor (including any Markdown editor) and have them synced automatically into your project – and vice versa.

Your documents in Scrivener must use markdown for bold / italic etc, but elements like footnotes and comments and lines between paragraphs can be automatically translated: E.g. all inline annotations in Scrivener are automatically converted to (( Inline Annotation )) and footnotes to {{ footnotes }} and back again.

Similarly, you can add new documents in the external editor, you can delete documents in the finder, and your project and the folder will remain in sync (you can even set it to take snapshots every time you sync, so you won’t lose data).

There are a few wrinkles, but it works really well when you’ve understood the process. I haven’t tried it with an obsidian vault, but on the face of it, there’s no obvious reason why it wouldn’t work – it does with every other markdown editor I’ve tried.

Obviously, you’d have to test out the features which matter to you most, but in principle you could use Scrivener for organisation, Scrivenings, citations etc, while doing most of the actual writing in Obsidian.

Once you’re ready to compile, the conversion from *italics* to italics can be done automatically. Once you’ve chosen your compilation format, just click the cog in the bar on the right hand side and tick the two ‘markdown’ options.

Just one other observation – don’t be concerned that you’re not using all the features of Scrivener – you’re not supposed to! I’d be surprised if anyone ever uses more than half of them… It’s a toolkit, not a straight-jacket, and many of those tools are intended for specific purposes and no-one needs them all.

[Bmosbacker]

@brookter This is extremely helpful. I never looked into the sync with external folder feature. This may give me the best of all worlds: 1) the ability to focus on text in an editor like iA Writer, the “linking my thinking” capabilities of Obsidian, and the power features of Scrivener. I’m going to give this a go and see how it works.

Thanks a million!

[brookter]

As I said, there are a few wrinkles to how it works and it may not fit your exact needs – I suspect linking may be something you’ll have to test thoroughly, for example – but it’s a powerful feature which isn’t all that well known, I think.

It’s worth giving the manual (14.3 Synchronised Folders) a skim first so you get the outline of how it’s supposed to work and the limitations.

Even if it’s not for this project, it’s a useful feature to know about. Good luck!

[svsmailus]

As much as I like and use Obsidian, I’ve found it’s easy to try and throw all text in it. I think it still wise to use the best tool for the job in hand. Obsidian will require far more of your time than Scrivener to accomplish the same task in Obsidian.

Obsidian is best at hyperlinked notes. That’s what I use it for.