I suffer from shiny idea syndrome, and this affliction riddles my task list with hundreds of great ideas, where some, like this essay, get completed, and others, like building a Commute Deck or my night-vision, perimeter trolling robot, probably won’t see the light of day any time soon. After discovering forgotten ideas, I decided I needed to refine my process.
This essay, contains musings on my file organization, and a follow-up essay contains the details, with the code I use.
Ode to a Text File
Before I begin, let’s chat nerd-to-nerd to establish a base-line. As an geek, working in text files containing code, you are good at working with text files; piping them, grepping them, and changing them with both editors and scripts. Boring old text files are a good medium storing my tasks and projects.
Over a dozen years ago, when I first started collecting text-based notes, I jumped on Gruber’s Markdown format, and used that for years. But Emacs started shipping with a package called Org (also called, org-mode), which, like Markdown, was a text-formatted style, but had a lot of special commands and features in Emacs. I wrote a script to convert all my Markdown files into org-mode, and have never looked back.
While the raw format of org-mode files isn’t as readable as markdown, within Emacs, I have customized my system (go fig) to look like a word processor. Not only are text formatting rendered in other font faces, hyperlinks show as a blue, underlined font, you can click or select to follow the link to a web site or another file on the system.
However, another advantage of a text-based solution is that you own your data. Sure, you can probably export your data from a third-party service like Evernote or Jira, but you’re still dependent on their EULAs, SLAs and whims. Sure, I’ve used file synchronization services like Dropbox, but you can move to Syncthing or any other service with a lot less effort.
Enough background. Thanks for putting up with my preaching the gospel of text.
On my workbench is a box. Actually, a few boxes. Each uncompleted project goes into a box… no lost screws. Sure unpacking and packing between each session is a hassle, but since the best projects take a few weekends, boxes are required.
Non-workbench-oriented projects go into mental boxes, and I’m thinking of my self organization in terms of boxes. Thoughts don’t go into physical boxes, but in my world, they live in text files. If you know me, you’ll know that my pensieve1 is a large collection of text files (specifically, Org formatted files readable by anything, and edited with Emacs and Orgzly).
Personal cloud storage mounted on all my devices is step one. Next, I divide my files into four workbenches (directories):
- Wanna remember something that doesn’t fit anywhere else
- Technical (and shareable) notes…essentially my engineering notebook
- Every task and todo goes into a box (file) in this directory
- Daily entries of events and completed projects
I initially put all my files in one large directory, findable with tags and searches, however, splitting into four boxes allows me to share each differently.
Memory boxes, like my recipes for vegetarian sushi, ends up in
my mental junk drawer,
personal; unless those notes are technical, like
all the ways to download code to an Arduino, go into
This division allows me to publish the entire directory to
Actual project boxes (files) go in the
projects directory, which allows
Orgzly to work with a smaller collection of files. If any project
garners images, they go into a directory of the same name. For instance:
-rw-rw-r-- 1 howard howard 520 May 27 09:54 after-school-class.org ... -rw-rw-r-- 1 howard howard 744 May 25 19:14 rpi-arcade.org drwxrwxr-x 2 howard howard 4096 May 20 10:58 rpi-arcade ...
If everything was just a note, placing things in boxes would be fine, however, much of what I jot are ideas, tasks or projects. In David Allen’s book, Getting Things Done, I like having an initial inbox where everything shiny and new gets initially placed, before appropriately processed and refiled. What follows is my flowchart for idea-to-completed project workflow:
Tasks and Smaller Projects
I like the idea of storing ideas in an inbox and then shuffle them around
text files as if they were paper index cards into boxes.
Since Orgzly (my mobile Org-focused organizing app) shows both the title of an
org file as well as the filename, I decided to name this initial box (inbox)
breathe.org, so listing my Notebooks immediately gives me perspective:
Each day, I review the contents of my inbox,
breathe.org and manipulate them
based on the workflow shown above. I describe the details (and code) for doing
this in a follow-up essay.
A critical part of storing information in files is being able to easily
search and retrieve that information. As a geek, my go to tool is often
ag), however those are line-oriented, and notes are often
seperated with line endings. Therefore, I need a search feature that is
Org supports tags in section headers, i.e. tags for each tasks, however, adding tags to top of the file, for example:
#+TAGS: recipe vegan dinner
Allows me to search in some unique ways.
Since I didn’t start tagging my file until after I moved hundreds of Markdown-formatted files from Notational Velocity to org-mode. To help, I wrote some tagging shell scripts, which I’ll update to be easier to call from Emacs.
Agendas and Dates
Tasks without a scheduled date are just wishes. Everything in
has a date, and choosing a date requires hitting
C-c C-s (or
, d s in
Spacemacs) Then Orgzly will give me a lovely agenda of today’s plan:
Why yes, that is the top section of my phone showing the Orgzly widget of my agenda on my home screen.
When I originally started jotting down the notes that became this essay, I was attempting to crystallize my thoughts where ideas and projects should live, but obviously grew into yet-another-proselytizing of Org and Emacs.
But upon reflection, I realized that I needed a workflow for processing my ideas into completed projects, and this encouraged a fun evening of Lisp hacking to facilitate. Since this essay is already too long, I wrote another essay containing an idea workflow…with lots of Lisp code.
My suggestion to you, gentle reader, is to (after converting your notes to Org) is to clarify what makes sense to you, and define your own organization and process. I would love to hear about your solution.
“This? It is called a Pensieve. I sometimes find, and I am sure you know the feeling, that I simply have too many thoughts and memories crammed into my mind.”
―attributed to Albus Dumbledore as recorded in Harry Potter and the Goblet of Fire.