💡

Update: Some things have changed in the past few years since writing the below text. I've documented them here.

As some of you who happen upon this post may know, I've been a full-time podcaster for a bit more than half a year now. It's fantastic, not least because I can focus fully on researching and writing my episodes. This, in turn, creates the interesting situation where I have more literature I'm going through than ever before, resulting in a bit of a chaos. Sure, I've had a host of different solutions to not lose total control, but I've never actually sat down and created a workflow that would incorporate all of those solutions into the one that would rule them all.

Fortunately, doing this full-time also meant I had more time on my hands than before, so I decided to tackle this issue and streamline the whole process.

And I've now, finally, arrived at a point, where I'm using one main tool to collect all my information and a plethora of smaller tools to facilitate getting all that information into that one tool.

After offering to share my workflow on my podcast, I realised that more people than anticipated have an interest in seeing how that works for me, so in order to streamline this process as well, I've decided to write it all up on here. This is not going to be short, but I'll try to keep it as concise as possible.

Let me first give you a quick overview of the tools I use, and then describe how I plug them all together to create my workflow.

One tool to rule them all

My main container, the place where all my notes, annotations, bookmarks and a host of other things end up is Obsidian. In its base version, it's a tool that lets you create a hierarchical structure of notes, which can be linked together wiki-style in a number of ways. It is vastly expandable, with literally thousands of plugins, most of them developed by a very active community.

You create a so-called vault, which in its essence is just a folder on your computer. In this folder, all the sub-folders and files are created, your settings and all your plugins are stored in this vault as well. It's a very flexible system, which in theory allows you to create countless very different workspaces, all of which you can switch between with a few mouse-clicks.

Every note I take, every book or article I annotate, every website that's of relevance to my research - it ends up in here. In order to faciliate the influx of all my notes and annotations into Obsidian, I'm using a few other tools which are just as important to this workflow.

A note about Markdown

Before we look at those other tools, there's one thing you should know:

Obsidian's editor works with Markdown. If you're not familiar with Markdown: it's a way of formatting text-files using a specific syntax, which can then be interpreted by an editor to look a certain way. Think word-processor without proprietary code. Instead you get a file that's readable by every text editor and which looks properly formatted in every Markdown editor.

Obsidian, like many other tools that focus on not walling you in, is interoperable with other Markdown editors. I can, for example, use Typora, a very clean looking editor that puts emphasis on allowing you to write unencumbered by huge menus or other cruft. I can then save those files back into Obsidian.

I like the idea of having all my files formatted with the Markdown syntax, because it means that even if I ever choose to switch editors, I will always have all my files, right there on my computer. For anyone who, like me, has been an Evernote user for ages, where exporting is possible but converting it all to a properly readable format is cumbersome, to say the least, this probably sounds attractive. With a vault filled with nothing but text-files, this will never be an issue in the future.

Of course, if you're used to writing your notes on, say, Evernote or OneNote, you'll be used to a rich-text editor. Meaning a toolbar or maybe even little pop-ups that let you do your formatting. Switching to a Markdown editor might be a bit of a drag, but once you get the hang of it, you'll love the ease with which you can quickly format your notes or texts. Hell, even Ghost, the blogging platform I'm using here, has Markdown support, and it makes writing this feel a lot more natural.

Well, let's move on to the rest of the tools.

Zotero

Anyone who's ever written a paper in the past ten years or so has probably at least heard of Zotero. It's a citation system, that allows you to save articles, books, websites or videos to its database. First and foremost, what it does save is the necessary metadata to then be able to export it with the proper citations. But it can also save PDFs or link to other files on a machine.

Calibre

While Zotero is great for articles, papers and sometimes books, my main library of e-books is managed by Calibre. It's an open-source tool and whenever I buy a new e-book, I import it into Calibre. Calibre too allows me to update the file with all the necessary metadata, and as you will see, it can also be linked to Zotero (and my e-book reader, but more about that later).

Zettlr

Zettlr is another Markdown editor, but one with a focus on writing. It is extensible, but even in its base version it's very useful as a tool to create longer texts, including bibliographies.

Zettlr is the tool I use to write my episodes. Both because it's very uncluttered, but also because it provides a handy feature that automatically creates my bibliographies. Hint: it also integrates well with Zotero, but more about that further below.

Syncthing

If you've been around the Internet for a while, you've probably used a number of services that sync your data between devices. Usually that means uploading all your data to a centralised server, which will then sync it to your various devices.

Syncthing - an open-source tool - does this, but without the centralised structure. It basically allows you to set it up on whatever device you wish and then decide which folders you want to sync to your other devices.

I use syncthing to sync my Obsidian vault to my phone, but as you will see later, I also use it to sync Calibre and Zotero files to my phone or e-book reader.

Cloudron

I like to fool-proof my setups, so even though syncthing works without a centralised server, I have set up a Syncthing installation on Cloudron. Cloudron is a tool that allows you to easily and quickly set up software on your own server (or your own VPS). A while ago I did an interview with one of its founders in my other podcast.

Readwise

Readwise is the odd one out here. It's not an open-source tool, and it's hosted on a centralised server. But, it's the tool that helps me capture excerpts and notes in physical media and then transfer them to Obsidian.

It works like this: when finding a passage in a physical book I'd like to keep, I use the Readwise mobile app to take a picture of that page. Readwise then exctracts the text via OCR, and lets me select whatever I want to keep. I can add my own notes to it, or just leave it like that. Readwise then adds the necessary metadata and saves it to my account.

The kicker here: it also provides integration, among other things, with Obsidian. So whenever I save something to my account, it is automatically saved to a folder in my Obsidian installation. And it's a rather elegant integration too, because it creates a single note for each book, and every subsequent highlight is added to that single note.

As an aside: Readwise integrates with a host of other services as well. I sometimes use Pocket - which is now part of Mozilla - and whenever I annotate articles from within Pocket, they're piped through to Obsidian as well.

The Workflow

Now that we've covered the necessary tools, let's have a look at how I use them in detail.

Digital articles and books

When I find articles and books for a story I'm researching, I usually do it from my browser. The one I use is Firefox, because it's simply the most trustworthy of them all, and also works pretty damn well. And Zotero provides integration by way of an extension.

So whenever I stumble upon something, I save it simply by clicking a button in my browser's toolbar. Note, though, that this usually only means saving the metadata, not the actual article or book. If the PDF of an article is freely available, it is saved to Zotero, if it isn't, it either saves an abstract or a PDF containing the abstract. Once I've actually bought the article or the book, Zotero lets me add the file - or a link to the file on my computer.

If it's an article, I add it to Zotero directly. If it's an e-book, it's time for Calibre.

After adding a book to Calibre, I make use of a plugin for Calibre that lets me export the metadata of books which I can then import into Zotero. In this case, I usually opt for creating a simple link to the book in my Calibre library, as opposed to saving it to Zotero directly. I wouldn't want to create duplicates all over the place. If you're wondering why I don't just add everything to Zotero anyway, the simple reason is that not every book I add to Calibre is a book I need for my podcast. Some are simply there for pleasure, and I wouldn't want to clutter up my Zotero database with those.

Websites, by the way, are stored by the Zotero plugin as snapshots. If you'd prefer them to be saved as PDFs, you can do that too. Not as automagically yet as I'd wish, but it's little hassle. Just install the Print friendly & PDF extension, and once you've saved the website to Zotero, turn it into a PDF with this extension and then add that PDF to the website's Zotero entry manually.

Sending them to my e-reader

Once I've got everything in Calibre or Zotero, it's time to actually read and annotate my literature. To make that happen, I use another extension for Zotero called Zotfile.

Zotfile allows for synchronisation between Zotero and a tablet, and since my e-reader is basically an Android tablet, I can make use of that feature.

My e-reader is an Onyx Boox Nova Air, which is a very lightweight Android tablet with an e-ink display and a custom launcher. With a little bit of trickery I can install apps from any app store, including the standard Google Play Store. Which comes in handy with the next part of my workflow.

Everything's connected

The thing is, I'm not sending my Zotero files to my e-reader directly. What I'm doing is the following: setting up Zotfile, I created a custom folder on my computer, into which the PDFs I'm planning to read on my e-reader will be sent.

With the above mentioned Syncthing, I'm synchonrising this folder to my e-reader. On my e-reader, I've told my library app to routinely scan this folder, and add whatever's in there to my library. I can then open these files on my e-reader and start highlighting and annotating.

Once I'm done annotating, I close the file, and Syncthing automatically syncs it back to my computer.

In Zotero, I then simply tell Zotfile to retrieve the file from my tablet again, which really is just that folder on my computer Syncthing is sending my files back to.

And now another magical thing happens:

Collecting the fruit of my labours

As soon as the file is retrieved by Zotfile, Zotero starts extracting highlights and annotations and saves them in a separate file. But that's not all.

With another Zotero plugin called MDNotes, I can then take these highlights and annotations and import them right into Obsidian (to find out how to properly set up MDNotes, this forum post proved invaluable to me).

EDIT: I found another plugin that is just a tad more automated than MDNotes. I wrote about it here.

But wait, there's more

Now, one of the core advantages of this whole process is that whatever I feed into Obsidian is already tagged with the correct metadata. For that to work, we need to look at two other plugins. One for Zotero, one for Obsidian.

The one for Zotero is called Better BibTex for Zotero. What this plugin helps me with in this scenario is to create an export of my entire Zotero library, including all the metadata. Better yet, it will always be automatically updated whenever I add new items to Zotero too. This export, which is a simple JSON file, is great, because it will help not just with Obsidian, but also with Zettlr (more about that later still).

Let's first look at that Obsidian plugin I talked about earlier. It's the citation for Obsidian plugin, and the name says it all. It takes as an input the above mentioned Zotero export in the form of that JSON file and thereby grants me access to my entire Zotero library from within Obsidian. It is highly customisable, and in its purest form lets you add in-line citations to texts and create separate files for each item in that library.

And that's where we go back to MDNotes. When I tell MDNotes from within Zotero to add highlights and annotations to Obsidian, it creates a file with those highlights and annotations complete with the item's metadata. The metadata - you guessed it - comes from the citation plugin, which in turn got it from the BibTex export.

As soon as that file is in Obsidian, I can, for example, move it into my research folder for that specific episode.

One more thing

If you're still reading, then by now you'll have gathered that between Readwise, Zotero and the MDNotes plugin I've managed to feed everything into Obsidian I need for my episodes, allowing me to start writing. You'll also remember that I mentioned Zettlr and its automatic bibliography-creation abilities.

The thing is, the citation plugin for Obsidian is great, but it doesn't create bibliographies automatically. If I want a list of all the literature I've used in an episode without manually putting it together, I need Zettlr.

And here's the beauty of it all: Zettlr uses the same file to get all of Zotero's metadata as the Obsidian plugin does. Which means that all the citations I've added to a file in Obsidian, will be recognised by Zettlr as well, and after importing the file will have Zettlr automatically create my bibliography.

And then, my friends, comes the last part of my workflow: the actual writing of an episode.

I open up Zettlr, and either import what I've already collected in Obsidian or open up a folder I've created within Obsidian to use as my workspace on Zettlr. All my imported highlights are there, all my literature is there and all the metadata is there as well.

Give or take five hours or so later, I'll have a fully formed episode, including an automatic bibliography.

Zettlr is great, because it also allows me to export whatever I've written in a plethora of formats. So if I, for whatever reason, need to export my text to, say, an HTML document, Zettlr does that for me, and automatically appends the file with my automatically created bibliography.

Your mileage may vary

So here we are. This is my workflow, and after sitting down and putting it all into words, it seems a tiny bit complex, but really, once the whole thing is set up, it works rather seamlessly. I do admit, it's very much tailored to me, and there might be a number of ways to do this differently as well. Call this workflow a snapshot, because if I find a better solution, I won't hesitate to change or amend it.

A few more clarifications which didn't really fit anywhere else:

Syncthing also syncs my Obsidian vault to my phone. I've opted to do this via the above mentioned Cloudron, simply because I don't always have my computer turned on. In order to not run into syncing anomalies, both my computer and my phone sync to and from Cloudron.
I also send e-books I read for pleasure to my reader via Syncthing. It works similarly to Zotfile. I simply create a folder which I then tell Calibre is the location of my e-reader. Whenever I then want Calibre to send a book to my device, the book is copied to that folder, and Syncthing does the rest.
I could cut out a few of these things if I used an e-reader like a Kindle, which has a webreader that lets you add annotations and highlights. The thing is, I don't want to use the Amazon ecosystem more than I already do. I also like to keep my e-book collection on my computer. Something which is, in theory, possible with a Kindle, if you use the plugin that lets you deDRM your files, etc. But still, it would be additional work, and I'm lazy.
I could also cut out Syncthing, at least for a few things, if I used the built in sync that's provided by the makers of Obsidian. I do try to keep as much of my data on my own computer, or at least my own server (or one I've rented), so I've decided against it. Syncthing, at least for the moment, does a good job doing what it's supposed to do.
A lot of the things I described here live on my computer. In order to avoid catastrophic failure of my setup through, say, a defective disk-drive, everything is also synced to my Nextcloud installation (which, conveniently enough, is hosted via the above described Cloudron).
If you're currently using Evernote and want to export your notes to Obsidian, there's a handy tool called YARLE that helps you with that.
Also, apart from Syncthing, every tool in my workflow is cross-platform. My OS of choice is Linux, but you should be able to replicate most of this on Windows and OSX/iOS. Except for Syncthing, which seems to have issues with iOs.
Obsidian is actually not an open source tool, but most plugins developed by the community are. There have been discussions in the past whether Obsidian will be open-sourced, but I don't think it's on the horizon just yet.

Final words

If you ever run into issues with Obsidian or Zotero or even Zettlr, the communities around these tools are amazingly vast and helpful. There's a ton of introductory videos on YouTube to explain what Obsidian is and what it can help you with. This holds true for just about every fairly popular Obsidian plugin out there.

To be honest, I feel like I'm only scratching the surface with what you can actually do with Obsidian. Its underlying philosophy is similar to that of a wiki, as it encourages and faciliates the connection of notes and thoughts to create something bigger than just the sum of its parts. Many people who have already devoted a lot more time to Obsidian can explain this to you in far greater and undoubtedly better detail, for example Josh Duffney, who in this post/newsletter describes the underlying methodology of the Zettelkasten system Obsidian is based on.

Setting up a workflow similar to mine might seem daunting (or overly complicated) at first, but if you spend some time looking into things, you'll realise that you can turn Obsidian, with the help of some additional tools, into whatever you want it to be.

The List and visualisation

Obsidian: notes, knowledge management
Zettlr: a Markdown editor environment, if you will
Zotero: litature and citation management
Zotfile: extended PDF management for Zotero
Syncthing: sync everything without the need for a central server
Readwise: save and sync highlights and annotations
Cloudron: host your own software, with one click