Script to convert OO to markdown, tex, and PDF using pandoc


#1

This script converts the current OmniOutliner document to markdown, tex, and then PDF via pandoc.

If you apply an appropriately named named style to a row – e.g. “Heading 1,” “Heading 2,” “Blockquote,” “Ordered List” – then the script will automatically add the relevant markdown tag e.g. “#” or “>”. Special thanks to @SGIII for the assist with this.

Unfortunately, the script does not currently convert styled text within a row to its markdown equivalent, so you’ll need to use markdown tags for bold and italic text (for example). I’m still trying to figure out how to convert a rich text object to markdown using javascript. I gather it will involve iterating over the attribute runs in the object.

The markdown, tex, and pdf files – as well as all the ancillary LaTeX files – are created in a new folder on the desktop called “LaTeX.” Requires pandoc to be installed, creation/installation of a custom LaTeX template for pandoc containing the necessary front- and back-matter to be installed (e.g. “\begin{document}”), and editing the script to reflect the name of that template. Note that OO document filenames with special characters such as parentheses currently break the script.

function run() {

// Name of your pandoc template file (don't include the file extension); you'll need one of these to include necessary LaTeX front- and back-matter e.g. "\begin{document}." (You'll also need to look up where pandoc stores these and put yours there.)

var pandocTemplate = 'articletemplate';

// Setup

var app = Application.currentApplication();
app.includeStandardAdditions = true;
var OmniOutliner = Application('OmniOutliner');

// Get the current document

var doc = OmniOutliner.documents[0];

// Get the name (stripped of spaces) of the Omni Outliner document. The script may fail if your filename includes certain characters, e.g. parentheses. (You could also just replace this with something like "paper.")

var fileName = doc.name().replace(/\s/g, '');

// Create a directory on the desktop to hold our new files

var desktopString = app.pathTo("desktop").toString()
app.doShellScript(`mkdir -p ${desktopString}/LaTeX/`);

// The text of the paper

var paperText = "";

// Loop through rows and append their text to paperText
// TODO: how to convert the rich text styling to markdown instead of ignoring it?

doc.rows().forEach(function(theRow) {
	if (Object.keys(theRow.style.namedStyles).length > 0) {
		switch(theRow.style.namedStyles[0].name()) {
			case "Heading 1":
				paperText += "# ";
				break;
			case "Heading 2":
				paperText += "## ";
				break;
			case "Heading 3":
				paperText += "### ";
				break;
			case "Blockquote":
				paperText += "> ";
				break;
			case "Ordered List":
				paperText += "1. ";
				break;
			case "Unordered List":
				paperText += "* ";
				break;
		}	
	}
	paperText += theRow.cells[1].richText();
	paperText += "\r\r";
});

// Convert the text of the paper to UTF8 encoding so pandoc can read it

paperText = $.NSString.alloc.initWithUTF8String(paperText);

// Write paperText to a new markdown file

var file = `${desktopString}/LaTeX/${fileName}.md`
paperText.writeToFileAtomicallyEncodingError(file, true, $.NSUTF8StringEncoding, null);

// Use pandoc to convert that markdown file to a tex file

shellCommand = `/usr/local/bin/pandoc ${desktopString}/LaTeX/${fileName}.md -f markdown -t latex -o ${desktopString}/LaTeX/${fileName}.tex --template=${pandocTemplate}`;

app.doShellScript(shellCommand);

// Compile our new tex file to PDF using xelatex

shellCommand = `/Library/TeX/texbin/xelatex --output-directory=${desktopString}/LaTeX/ ${desktopString}/LaTeX/${fileName}.tex`;

app.doShellScript(shellCommand);

return true;

}

// From apple's documentation for Javascript for Automation
 
function writeTextToFile(text, file, overwriteExistingContent) {
    try {
 
        // Convert the file to a string
        var fileString = file.toString()
 
        // Open the file for writing
        var openedFile = app.openForAccess(Path(fileString), { writePermission: true })
 
        // Clear the file if content should be overwritten
        if (overwriteExistingContent) {
            app.setEof(openedFile, { to: 0 })
        }
 
        // Write the new content to the file
        app.write(text, { to: openedFile, startingAt: app.getEof(openedFile) })
 
        // Close the file
        app.closeAccess(openedFile)
 
        // Return a boolean indicating that writing was successful
        return true
    }
    catch(error) {
 
        try {
            // Close the file
            app.closeAccess(file)
        }
        catch(error) {
            // Report the error is closing failed
            console.log(`Couldn't close file: ${error}`)
        }
 
        // Return a boolean indicating that writing was successful
        return false
    }
}

Still interested in Markdown support in OmniOutliner
Translating Illustrations: JXA or OmniJS, or something else entirely?
Still interested in Markdown support in OmniOutliner
#2

@dmgrant

Thanks much for posting this. Though over my head in spots, it’s really useful to be able to study a live example of JXA with OO.

Re converting the bold and italic in an rtf object to markdown, perhaps the command line tool textutil could help. In AppleScript, for example, if the rich text is on the system clipboard it can be converted to html with something like this:

set the clipboard to (the clipboard as «class RTF ») set theHTML to do shell script "pbpaste -Prefer rtf | textutil -convert html -stdin -stdout"

I don’t know much about markdown and am thinking aloud here, but assume extracting the bold and italic tags and their contents from the html wouldn’t be that hard.

SG


#3

@dmigrant:

Here’s a working example in AppleScript, based on a script posted by Rob Trew.

-- marks up strong and emphasis in topic of a single selected OO row

tell application "OmniOutliner"
	tell front document
		tell first selected row
			tell its topic's attribute runs
				set {lstText, lstFont} to {its text, its font}
			end tell
		end tell
	end tell
end tell
--return {lstText, lstFont}

set outTxt to ""
repeat with i from 1 to lstText's length
	set {aChunk, aFont} to {lstText's item i, lstFont's item i}
	if aFont contains "bold" or aFont contains "black" then
		set outTxt to outTxt & "**" & aChunk & "** "
	else if lstFont's item i contains "italic" then
		set outTxt to outTxt & "*" & aChunk & "*"
	else
		set outTxt to outTxt & aChunk
	end if
end repeat
return outTxt

SG


#4

@dmgrant

Here’s a direct translation of the AppleScript. Probably more “JavaScripty” ways of doing this. Need to incorporate possibility of Bold instead of Black in some font names.

SG

    oo = Application('OmniOutliner');
    var activeDoc = oo.documents[0];
    var theRow = activeDoc.selectedRows[0];
    var attribRuns = theRow.topic.attributeRuns;
    var outTxt = "";


    for (var i=0; i < attribRuns.length; i++) {
      var aChunk = attribRuns[i].text();
      var aFont = attribRuns[i].font();
      if((aFont.includes('Bold')) || (aFont.includes('Black')) ) {
            outTxt += "**" + aChunk + "**";
         } else if (aFont.includes('Italic')) {
             outTxt += "*" + aChunk + "*"; 
         } else {
          outTxt += aChunk;
         }
      }

    outTxt

#5

Well, whatever works, and there are many approaches :-) FWIW I tend to write in this kind of idiom (for Sierra ES6 JS)

(() => {
    'use strict';

    // GENERIC ----------------------------------------------------------------

    // enumFromTo :: Int -> Int -> [Int]
    const enumFromTo = (m, n) =>
        Array.from({
            length: Math.floor(n - m) + 1
        }, (_, i) => m + i);

    // OmniOutliner row to text with MD emphases ------------------------------

    // rowTextMD :: OO.Row -> String
    const rowTextMD = row => {
        const
            as = row.topic.attributeRuns;
        return enumFromTo(0, as.length - 1)
            .reduce((s, i) => {
                const
                    attrib = as.at(i),
                    fnt = attrib.font(),
                    bld = (fnt.includes('Bold') || fnt.includes('Black')) ? (
                        '**'
                    ) : '',
                    ital = fnt.includes('Italic') ? '*' : '';
                return s + bld + ital + attrib.text() + ital + bld;
            }, '') + '\n';
    };


    // TEST -------------------------------------------------------------------
    const
        ds = Application('OmniOutliner')
        .documents,
        d = ds.length > 0 ? ds.at(0) : undefined;

    return (d ? d.selectedRows() : [])
        .reduce((s, row) => s + rowTextMD(row), '');
})();

And if you need ES5 JS (for pre-Sierra OS X), you the Babel REPL will back-convert for you to something like:

(function () {
    'use strict';

    // GENERIC ----------------------------------------------------------------

    // enumFromTo :: Int -> Int -> [Int]
    var enumFromTo = function enumFromTo(m, n) {
        return Array.from({
            length: Math.floor(n - m) + 1
        }, function (_, i) {
            return m + i;
        });
    };

    // OmniOutliner row to text with MD emphases ------------------------------

    // rowTextMD :: OO.Row -> String
    var rowTextMD = function rowTextMD(row) {
        var as = row.topic.attributeRuns;
        return enumFromTo(0, as.length - 1).reduce(function (s, i) {
            var attrib = as.at(i),
                fnt = attrib.font(),
                bld = fnt.includes('Bold') || fnt.includes('Black') ? '**' : '',
                ital = fnt.includes('Italic') ? '*' : '';
            return s + bld + ital + attrib.text() + ital + bld;
        }, '') + '\n';
    };

    // TEST -------------------------------------------------------------------
    var ds = Application('OmniOutliner').documents,
        d = ds.length > 0 ? ds.at(0) : undefined;

    return (d ? d.selectedRows() : []).reduce(function (s, row) {
        return s + rowTextMD(row);
    }, '');
})();

#6

Many thanks to you both! I’ve been trying to figure this out for a long time. Here’s the completed script, now with support for markdown bold and italic tags!

function run() {

// My pandoc template file; you'll need one of these to include necessary LaTeX front- and back-matter e.g. "\begin{document}." (You'll also need to look up where pandoc stores these and put yours there.)

var pandocTemplate = 'dmgarticletemplate';

// Setup

var app = Application.currentApplication();
app.includeStandardAdditions = true;
var OmniOutliner = Application('OmniOutliner');

// Get the current document

var doc = OmniOutliner.documents[0];

// Get the name (stripped of spaces) of the Omni Outliner document. The script may fail if your filename includes certain characters, e.g. parentheses.

var fileName = doc.name().replace(/\s/g, '');

// Create a directory on the desktop to hold our new files

var desktopString = app.pathTo("desktop").toString()
app.doShellScript(`mkdir -p ${desktopString}/LaTeX/`);

// The text of the paper

var paperText = "";

// Loop through rows and append their text to paperText

doc.rows().forEach(function(theRow) {
	if (Object.keys(theRow.style.namedStyles).length > 0) {
		switch(theRow.style.namedStyles[0].name()) {
			case "Heading 1":
				paperText += "# ";
				break;
			case "Heading 2":
				paperText += "## ";
				break;
			case "Heading 3":
				paperText += "### ";
				break;
			case "Blockquote":
				paperText += "> ";
				break;
			case "Ordered List":
				paperText += "1. ";
				break;
			case "Unordered List":
				paperText += "* ";
				break;
		}	
	}
	paperText += rowTextMD(theRow);
	paperText += "\r\r";
});

// Convert the text of the paper to UTF8 encoding so pandoc can read it

paperText = $.NSString.alloc.initWithUTF8String(paperText);

// Write paperText to a new markdown file

var file = `${desktopString}/LaTeX/${fileName}.md`
paperText.writeToFileAtomicallyEncodingError(file, true, $.NSUTF8StringEncoding, null);

// Use pandoc to convert that markdown file to a tex file

shellCommand = `/usr/local/bin/pandoc ${desktopString}/LaTeX/${fileName}.md -f markdown -t latex -o ${desktopString}/LaTeX/${fileName}.tex --template=${pandocTemplate}`;

app.doShellScript(shellCommand);

// Compile our new tex file to PDF using xelatex

shellCommand = `/Library/TeX/texbin/xelatex --output-directory=${desktopString}/LaTeX/ ${desktopString}/LaTeX/${fileName}.tex`;

app.doShellScript(shellCommand);

return true;

}

// From apple's documentation for Javascript for Automation
 
function writeTextToFile(text, file, overwriteExistingContent) {
    try {
 
        // Convert the file to a string
        var fileString = file.toString()
 
        // Open the file for writing
        var openedFile = app.openForAccess(Path(fileString), { writePermission: true })
 
        // Clear the file if content should be overwritten
        if (overwriteExistingContent) {
            app.setEof(openedFile, { to: 0 })
        }
 
        // Write the new content to the file
        app.write(text, { to: openedFile, startingAt: app.getEof(openedFile) })
 
        // Close the file
        app.closeAccess(openedFile)
 
        // Return a boolean indicating that writing was successful
        return true
    }
    catch(error) {
 
        try {
            // Close the file
            app.closeAccess(file)
        }
        catch(error) {
            // Report the error is closing failed
            console.log(`Couldn't close file: ${error}`)
        }
 
        // Return a boolean indicating that writing was successful
        return false
    }
}

// Code below written by draft8, based on code written by SGIII, in turn adapted from AppleScript code written by Rob Trew

const rowTextMD = row => {
        const
            as = row.topic.attributeRuns;
        return enumFromTo(0, as.length - 1)
            .reduce((s, i) => {
                const
                    attrib = as.at(i),
                    fnt = attrib.font(),
                    bld = (fnt.includes('Bold') || fnt.includes('Black')) ? (
                        '**'
                    ) : '',
                    ital = fnt.includes('Italic') ? '*' : '';
                return s + bld + ital + attrib.text() + ital + bld;
            }, '') + '\n';
    };
	
// enumFromTo :: Int -> Int -> [Int]
const enumFromTo = (m, n) =>
	Array.from({
	length: Math.floor(n - m) + 1
    }, (_, i) => m + i);

#7

Looks good :-)

If the translation directions multiply, it can be good to use JSON / JS Objects as a hub for nested texts (outlines), something like this:

Rough two stage example below:

  1. OmniOutliner to [Node {text:String, nest:[Node], ...}]
  2. [Node {text:String, nest:[Node], ...}] to Markdown, (with same options as iThoughts exporter)
var strClip = (() => {
    'use strict';

    // OmniOutliner to TEXT-NEST, then TEXT-NEST to MARKDOWN

    // example of format translation through a textNest hasSubTopics

    // Rough draft ver 0.05
    // 0.05 -- Moved extra linefeed from after hash header to before
    // 0.04 -- Simplified ooRowsJSO
    // 0.03 -- Slightly faster version of cellTextMD – fewer AE events

    // Copyright(c) 2017 Rob Trew
    //
    // Permission is hereby granted, free of charge, to any person obtaining a
    // copy of this software and associated documentation files(the "Software"),
    // to deal in the Software without restriction, including without limitation the rights
    // to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
    // copies of the Software, and to permit persons to whom the Software is
    // furnished to do so, subject to the following conditions:
    //
    // The above copyright notice and this permission notice shall be included in all
    // copies or substantial portions of the Software.
    //
    // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
    // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    // SOFTWARE.

    // GENERIC ----------------------------------------------------------------

    // any :: (a -> Bool) -> [a] -> Bool
    const any = (f, xs) => xs.some(f);

    // concat :: [[a]] -> [a] | [String] -> String
    const concat = xs =>
        xs.length > 0 ? (() => {
            const unit = typeof xs[0] === 'string' ? '' : [];
            return unit.concat.apply(unit, xs);
        })() : [];

    // curry :: Function -> Function
    const curry = (f, ...args) => {
        const go = xs => xs.length >= f.length ? (f.apply(null, xs)) :
            function () {
                return go(xs.concat(Array.from(arguments)));
            };
        return go([].slice.call(args, 1));
    };

    // flip :: (a -> b -> c) -> b -> a -> c
    const flip = f => (a, b) => f.apply(null, [b, a]);

    // isInfixOf :: Eq a => [a] -> [a] -> Bool
    const isInfixOf = (needle, haystack) =>
        haystack.includes(needle);

    // log :: a -> IO ()
    const log = (...args) =>
        console.log(
            args
            .map(show)
            .join(' -> ')
        );

    // min :: Ord a => a -> a -> a
    const min = (a, b) => b < a ? b : a;

    // replicate :: Int -> a -> [a]
    const replicate = (n, a) => {
        let v = [a],
            o = [];
        if (n < 1) return o;
        while (n > 1) {
            if (n & 1) o = o.concat(v);
            n >>= 1;
            v = v.concat(v);
        }
        return o.concat(v);
    };

    // replicateS :: Int -> String -> String
    const replicateS = (n, s) => concat(replicate(n, s));

    // show :: a -> String
    const show = x => JSON.stringify(x, null, 2);

    // zipWith :: (a -> b -> c) -> [a] -> [b] -> [c]
    const zipWith = (f, xs, ys) =>
        Array.from({
            length: min(xs.length, ys.length)
        }, (_, i) => f(xs[i], ys[i]));

    // JSO TEXT-NEST TO MARKDOWN ----------------------------------------------

    // jsoMarkdown :: Dictionary -> [Node] -> String
    const jsoMarkdown = (dctOptions, xs) => {
        const
            indent = dctOptions.indent || '\t',
            headerLevels = dctOptions.headerLevels || 3,
            listMarker = dctOptions.listMarker || '-',
            notesIndented = dctOptions.notesIndented || true;
        const
            jsoNodeMD = curry((intLevel, strIndent, node) => {
                const
                    blnHash = intLevel <= headerLevels,
                    index = node.number,
                    strNum = ((blnHash || (index === undefined)) ? (
                        ''
                    ) : index + '.'),
                    strPrefix = (blnHash ? (
                        replicateS(intLevel, '#')
                    ) : strNum || listMarker) + ' ',
                    noteIndent = notesIndented ? strIndent + indent : strIndent,
                    nextIndent = blnHash ? '' : indent + strIndent,
                    note = node.note,
                    strNotes = (note !== '') ? (
                        note.split('\n')
                        .map(x => noteIndent + x)
                        .join('\n') + '\n\n'
                    ) : '',
                    nest = node.nest;
                return (blnHash ? '\n' : '') +
                    strIndent + strPrefix + node.text + '\n' +
                    strNotes + (nest.length > 0 ? (
                        nest.map(jsoNodeMD(intLevel + 1, nextIndent))
                        .join('') + '\n'
                    ) : '');
            });

        return xs.reduce((a, x) => a + jsoNodeMD(1, '', x) + '\n', '');
    };

    // OMNI-OUTLINER TO JSO / JSON --------------------------------------------

    // Either with or without (see blnMD) MarkDown emphases for Bold/Italic
    // ooRowsJSO :: Bool -> OO.Document -> [Node {text: String, nest: [Node]]}
    const ooDocJSO = (blnMD, doc) => [{
        text: '[' + doc.name() + '](file:://' + Path(doc.file())
            .toString() + ')',
        nest: ooRowsJSO(blnMD, doc.children)
    }];

    // ooRowsJSO :: Bool -> OO.Rows -> Node {text: String, nest: [Node]}
    const ooRowsJSO = (blnMD, rows) =>
        rows()
        .map((r, i) => ({
            text: blnMD ? cellTextMD(r.topicCell) : r.topic(),
            number: r.style.attributes
                .byName('heading-type(com.omnigroup.OmniOutliner)')
                .value() !== 'None' ? i + 1 : undefined,
            note: blnMD ? cellTextMD(r.noteCell) : r.note(),
            nest: r.hasSubTopics ? ooRowsJSO(blnMD, r.children) : []
        }));

    // contains :: String -> String -> Bool
    const contains = curry(flip(isInfixOf));

    // cellTextMD :: OO.Cell -> String
    const cellTextMD = cell => {
        const as = cell.richText.attributeRuns;
        return zipWith((txt, fnt) => {
                const
                    bld = any(contains(fnt), ['Bold', 'Black']) ? (
                        '**'
                    ) : '',
                    ital = isInfixOf('Italic', fnt) ? '*' : '';
                return bld + ital + txt + ital + bld;
            }, as.text(), as.font())
            .join('');
    };

    // TEST -------------------------------------------------------------------
    const ds = Application('OmniOutliner')
        .documents,
        d = ds.length > 0 ? ds.at(0) : undefined;

    // Edit optional values for indent, headerLevels, notesIndented, listMarker
    return d ? jsoMarkdown({
        indent: replicateS(4, ' '), // or '\t'
        headerLevels: 3,
        notesIndented: true,
        listMarker: '-' // or '*' or '+'
    }, ooDocJSO(true, d)[0].nest) : '';
})();

var a = Application.currentApplication(),
    sa = (a.includeStandardAdditions = true, a);

sa.setTheClipboardTo(strClip);

strClip

Still interested in Markdown support in OmniOutliner
#8

Wow, this is terrific. Thank you!

Also, I’m now realizing that what I thought were three extraordinarily helpful sources of information about Mac OS automation (Rob Trew, complexpoint, draft8) are really one source. Thanks for all of your generosity helping the rest of us figure this stuff out…


#9

What I haven’t yet figured out is how to do this with the rather wonderful omniJS (fast, iOS as well as Mac, simpler API) – specifically, I’m not sure whether the omniJS model yet allows us to return a value (like a JSO text Nest or JSON string ) to another (non-OmniApp) context.

In TaskPaper’s (also excellent) internal JS Context, we can return a value through JXA/Applescript, to which a document.Evaluate(jsString) method has been added. I haven’t yet quite fathomed whether the OmniJS URL.call() method for example would allow the JS-enabled iOS Drafts or 1Writer, (or macOS anything scriptable), for example, to submit a URL to an omniJS app, and get back a url-encoded JSON string which could be rewritten to MD, TaskPaper, etc


#10

PS I’ve updated the script above with a fractionally faster version of cellTextMD

(batch reading texts and fonts of an attributeRuns collection, for fewer Apple Events)


#11

Footnote – a version of the cellTextMD function which picks up inline links as Markdown links, and also catches fonts with either ‘Oblique’ or ‘Italic’ in their name for the single-star Markdown emphasis.

    // cellTextMD :: OO.Cell -> String
    const cellTextMD = cell => {
        const as = cell.richText.attributeRuns;
        return zipWith3((txt, fnt, style) => {
                const
                    bld = any(contains(fnt), ['Bold', 'Black']) ? (
                        '**'
                    ) : '',
                    ital = any(contains(fnt), ['Oblique', 'Italic']) ? (
                        '*'
                    ) : '',
                    url = style.attributes.byName('link')
                    .value();
                return bld + ital + (
                    isNull(url) ? txt : ('[' + txt + '](' + url + ')')
                ) + ital + bld;
            }, as.text(), as.font(), as.style())
            .join('');
    };

It uses a generic zipWith3(f, xs, ys, zs) function, which can be defined as:

    // zipWith3 :: (a -> b -> c -> d) -> [a] -> [b] -> [c] -> [d]
    const zipWith3 = (f, xs, ys, zs) =>
        Array.from({
            length: Math.min(xs.length, ys.length, zs.length)
        }, (_, i) => f(xs[i], ys[i], zs[i]));

Apart from isNull() the other generic functions are given in the original listing in this thread

    // isNull :: [a] | String -> Bool
    const isNull = xs =>
        Array.isArray(xs) || typeof xs === 'string' ? (
            xs.length < 1
        ) : undefined;