Daniel Beer Atom | RSS | About

Markdown-based organizer

2 May 2020 (updated 3 Jan 2022)

After trying a lot of different calendar and organizer programs I finally gave up and wrote my own. This script enables me to summarize and organize information by topic using a plain text format spread across multiple files. On request, all files are scanned for tags indicating TODO items or deadlines, and the result is printed in a short half-page hierarchial summary.

The input file format is markdown-like and is parsed into a tree structure by the script. Two levels of headings are recognized, underlined by either === or ---. Hierarchical lists are recognized by indentation and the presence of bullets (* or -). The first line of any item is treated as a short label. Other lines are ignored, unless they contain a special directive.

Items can contain directive lines of the following type:

• INCLUDE <file>: this causes <file> to be attached at this point. The inclusion happens after parsing, so that items at the root of the included file appear at the level where the directive was found, and this doesn’t depend on particular formatting choices.

• TODO: this marks an item as a TODO item, to be included in the summary output.

• DATE <YYYY-MM-DD>: this marks a deadline. Items with deadlines are included in the summary output if they are within the horizon period (normally 14 days from the current day), or if the deadline has passed.

• PROJECT: this marks a node as the root of a project.

• PRIORITY <n>: set the priority of a node. The --prioritize option can then be used to prune all but the highest-priority items within each project.

• [<n>]: numbers in square brackets indicate a cost estimate. Cost estimates found in all subtrees marked TODO are summed and the total cost for each subtree is displayed in the output. If a node has both a cost and a TODO directive, then the TODO must come first.

Get the script here:

• organize.pl

It has no dependencies aside from a Perl interpreter. Run with --help to get a list of options:

Usage: /usr/local/bin/organize.pl [options] [<file1> ...]

Options may be any of:
  --help         Show this text.
  --version      Print version and exit.

  --horizon <H>     Set horizon, in days.
  --agenda <A>      Set agenda length, in days.
  --no-color        Do not use colour codes.
  --prioritize <P>  Prune to <P> levels of priority within each project.

Example

Suppose we create a file called example.txt, with the following contents:

This is an example file showing various features of organize.pl.

Projects
========

  * Foo
    The contents of "foo/TODO" will be attached at this point in the tree.
    INCLUDE foo/TODO

    You can mix inclusion with inline items easily:
    - TODO Example item

  * Send invoices
    DATE 2020-05-31

Home
====

  * TODO Insulate living room floor

  * Rates due
    DATE 2020-08-15

Website
=======

  * TODO Write article on organize.pl
    - TODO Write up some example files
    - ansi2html can be used to produce text "screenshots"

  * Renew SSL certificate
    DATE 2020-09-04

The script will look for a file called foo/TODO (where the path is relative to the file containing the directive). This file contains:

Here are some notes on the foo project

  * TODO Firmware changes

  * Meeting with Bob
    DATE 2020-05-04

Board changes
=============

We need the following changes for revision 2:

  * TODO R13 incorrectly specified

  * TODO Get new quote for stencil

  * Some other changes.
    This note is parsed by organize.pl, but not included in the summary
    because it has no tags.

Web interface
=============

  * TLS certificate needs renewing
    DATE 2020-05-06

Running the script on the root file results in the following output:

Implementation

The script is divided into four packages:

• Day: this contains functions for parsing and printing ISO-8601 dates. Dates are represented internally as an integer indicating the number of days elapsed since an arbitrary epoch.

• Multitree: this is our Markdown parser. It reads a file and returns a tree of lines of text. It parses INCLUDE directives and attaches included files at the appropriate point in the tree.

• Org: this is our high-level parser. It takes the tree from the Markdown parser and looks for TODO and DATE directives, and adds tags to tree nodes when it finds them. It also contains functions for manipulating tree nodes and printing summary output.

• main: this parses command-line arguments, reads input files, and runs through the filtering stages.

The output of the Multitree parser is an array reference containing top-level lines. Indented lines are represented recursively by array references. Lines are grouped up until the next line with a bullet point or a change of hierarchy level. The parser yields the following for our example above:

[
  'This is an example file showing various features of organize.pl.',
  '',
  [
    'Projects',
    '',
    [
      'Foo',
      'The contents of "foo/TODO" will be attached at this point in the tree.',
      'Here are some notes on the foo project',
      '',
      [
        'TODO Firmware changes',
        ''
      ],
      [
        'Meeting with Bob',
        'DATE 2020-05-04',
        ''
      ],
...

The parsed lines for each input file and then processed into a tree by the Org package. Each node in the tree is a hash reference with the following fields:

• Lines: an array reference containing the original text from which this node was composed.

• Title: the title of this node, to appear in summary output.

• Todo: if present, this item has a TODO tag.

• Date: if present, this item has a deadline, and the value is the parsed deadline date.

• Children: an array reference containing child nodes.

A pseudo-node at the top-level with no content is created to join all input files into one tree.

The tree is then filtered based on the Todo and Date fields. Nodes which have a non-zero Todo field, or a Date which is within the configured horizon are marked with a Keep attribute. Nodes with a Keep attribute and their ancestors are retained – all other nodes are removed.

A squash phase is run to collapse spurious hierarchy levels for nodes with only one child. Siblings are sorted by deadline date. Finally, we print a hierachical summary of the remaining items in the tree, and a timeline summary.

Licence

Copyright (C) 2020 Daniel Beer <dlbeer@gmail.com>

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.