Translating Illustrations: JXA or OmniJS, or something else entirely?

I’m really sorry for not making myself more clear in my post, I see now that I made this much too complicated by adding all the detail. So it got totally lost when clarified this exact thing in my second reply to you (14 days ago), when I wrote:

But what is blocking my progress is actually bugs in OmniGraffle, which I reported as:

  • OG #2173060 “Problems with JXA accessing items in OmniGraffle documents” (sent about a week ago, nothing but an auto-reply so far)
  • OG #2166190 “Omnigraffle: Export and JXA” (about a week ago) (which is very likely related to a problem I reported as OG #1820090 “OmniGraffle document model” (August 2017)

I had already tried the one thing at a a time approach with the export problem, see this other thread, but nobody replied. That post should contain the info you need to understand that particular problem.

About the shared layers, I’m pretty sure they are broken in JXA, see OG #2173060 for the details about that. From what you explain, things should be different in OmniJS, so I tried again. On my first test I was able to access the item in a shared layers, but the moment I tried accessing the second graphic in that layer (which does not exist), I also could no longer access the first. Then relaunched OmniGraffle and tried again, and now I could trigger this right away:

I’m a bit tired at the moment, so please forgive me if I missed anything obvious, but it appears to me that while I can access both objects in the layer that is not shared, I cannot access the single object in the shared layer. And the error message is the same as the one I described in the post above. I did not run any JXA, or a plugin or similar. The test file is the one I used in the post above.

When I attempt to copy the circle in the shared layer as JavaScript, I get this:

// Floating point values in this script may be rounded, resulting in minor visual differences     from the original
var canvas = document.windows[0].selection.canvas;
var g1 = canvas.newShape();

However, when I copy a circle in the first layer (not shared), I get this:

// Floating point values in this script may be rounded, resulting in minor visual differences from the original
var canvas = document.windows[0].selection.canvas;
var g1 = canvas.newShape();
g1.textAlongPathGlyphAnchor = 0;
g1.text = "";
g1.textVerticalPadding = 5;
g1.fillType = FillType.Solid;
g1.plasticHighlightAngle = null;
g1.cornerRadius = 0;
g1.flippedVertically = false;
g1.name = null;
g1.blendColor = null;
g1.textRotation = 0;
g1.shape = "Circle";
g1.fillColor = Color.RGB(1.0, 1.0, 1.0);
g1.strokeCap = LineCap.Round;
g1.textUnitRect = new Rect(0.10, 0.15, 0.80, 0.70);
g1.tripleBlend = false;
g1.gradientCenter = new Point(0.00, 0.00);
g1.strokePattern = StrokeDash.Solid;
g1.geometry = new Rect(125.00, 82.00, 316.00, 288.00);
g1.textRotationIsRelative = true;
g1.shadowColor = null;
g1.shadowFuzziness = 3;
g1.notes = "";
g1.locked = false;
g1.allowsConnections = true;
g1.strokeJoin = LineJoin.Round;
g1.imageOpacity = 0;
g1.blendFraction = 0;
g1.flippedHorizontally = false;
g1.rotation = 0;
g1.imageScale = new Size(0.00, 0.00);
g1.textColor = Color.black;
g1.automationAction = [];
g1.autosizing = TextAutosizing.Overflow;
g1.magnets = [];
g1.textVerticalPlacement = VerticalTextPlacement.Middle;
g1.strokeType = StrokeType.Single;
g1.actionURL = null;
g1.gradientAngle = 90;
g1.gradientColor = Color.RGB(0.20000000298023224, 0.20000000298023224, 0.20000000298023224);
g1.strokeColor = Color.RGB(0.0, 0.0, 0.0);
g1.image = null;
g1.userData = {};
g1.imagePage = 0;
g1.textSize = 16;
g1.shadowVector = new Point(0.00, 2.00);
g1.textWraps = true;
g1.plasticCurve = null;
g1.strokeThickness = 1;
g1.alignsEdgesToGrid = true;
g1.textHorizontalAlignment = HorizontalTextAlignment.Center;
g1.textHorizontalPadding = 5;
g1.imageSizing = ImageSizing.Manual;
g1.fontName = "HelveticaNeue";
g1.imageOffset = new Point(0.00, 0.00);

I’m curious if you can confirm this on your machine.

Yes, we agree that the things you reported are broken in JXA and it’s an active bug. Yes, I get the same result when I Copy as JavaScript on a shared layer, and that is because this bug goes far beyond with just JXA. Sorry that it took me some time to figure out what was going wrong. I will get these issues reported to the team and make sure I update the scope of the initial bug you reported. You can’t get anything from a shared layer at all, which is the single reason you are blocked. Let me work on this here and see if I can find anything out. I will talk to some people on Monday and see what I can learn.

Here is my revised script for figuring out what is going wrong with getting the text out of OmniGraffle. I run this in the console to see what is returned for the graphics in the frontmost document:

var groups = new Array()
     canvases.forEach(function(cnvs){
     	cnvs.graphics.forEach(function(graphic){
     		if(graphic instanceof Group){groups.push(graphic)}
     		 console.log(graphic + ' is Graphic ' + graphic.name + ' has text ' + graphic.text)
     	})
     })

Next week, by Friday, I am going to send you an email including some ideas on how you might proceed after I talk to some colleagues. Please keep in mind that I am not a support professional, which is why I am working on this on the weekend, as I still have to do my regular work. I use the OmniGraffle JavaScript library and I’d like to see us improve it. Please be patient with our support humans. Fewer of us code in JavaScript here than in other languages, so it does take us longer than a typical question to answer.

Thanks,
Lanette

Lanette,

my sincere thanks for your continued support, for sticking with this even though I probably did not make it easy for you, and for working on this in your free time, even though this is not your job. I appreciate what you do, and at the same time I feel this is not the way it should be, it’s not a sustainable way for an organization, and there’s obviously an underlying structural problem that needs to be addressed. Good luck with that.

I gather you are working as a tester, so may I in return suggest something that would probably make your life on your regular job a bit easier: both JXA and OmniJS lend themselves for automated testing, and basically that’s what I did when identifying those bugs: I created a test document that contained all the features I used in my illustrations (shared layers, formatting etc.), and then I wrote code to see how to access those features. It’s pretty simple to use that idea and create a suite of automated test that runs on your CI system and makes sure all the parts of the OmniJS API are up and running. For OmniJS, the obvious way to trigger tests is Script Links. For JXA, you can run osascript from the command line. For JXA you could even use Espresso through node-jxa.

The next update to OmniGraffle (v7.11) has fixes for accessing graphics on shared layers both from Omni Automation (OmniJS) and from Apple Events (including AppleScript and JXA).

I anticipate that test builds of OmniGraffle 7.11 will be available within the next week or so.

1 Like

Test builds with these fixes are available now!

https://omnistaging.omnigroup.com/omnigraffle/

Please let us know if you encounter scripting bugs in these builds; at this point in the test cycle, we should be able to turn around additional scripting fixes fairly quickly.

1 Like

First test reveals: export is still broken in 7.11 test (v196.1 r330841).

e.g. this

is not the same as this

or this

is not like this:

As the forum does not allow me to upload a zip, I’ll send it over via email so you can test for yourself.

Hi,

I created a small OmniJS API test suite to check for the stuff I was aware of, and it appears that while I can access graphics in shared layers now via iterating through the graphics collection of that layer, canvas.graphicWithName() appears to be broken in shared layers.

You can download the test suite and the document it uses and see for your self. Simply install the plugin, open the document and run the plugin. Look at the console to see test output.

Can you confirm this is broken, or was there something I missed?

I have not tested all this in JXA yet, just in OmniJS, and here’s some other bugs and glitches I found:

  • export via JXA still does not work (see my export script)
  • Omnigraffle Undo does not track changing object names
    • OmniGraffle is quite slow to update a plugin, it frequently happened to me that I triggered the tests after replacing the plugin, and OmniGraffle still ran the old plugin code
  • graphics in subgraphs can be accessed both via graphics and subgraphics collections. I think that is weird, and the array subgraphics is unnecessary. That’s the one place I am aware of where a graphics collection has a different name.

Thanks for reporting the problem! It might be helpful to split these issues into separate threads, so the circumstances are more clear. I agree that this shows that export is broken, but I was left guessing as to how you got that set of results: export seemed to be working fine in my own tests.

Since my tests were working and yours were not, even with the same input document (thank you for providing that in your github repository!), I did some digging to try to track down why my script was working when yours was not.

It looks like these results came from your JXA export script, and the rest of this post is based on that assumption. If I’m wrong about that, please let me know!

The difference between your script and my quick test was that your script was immediately closing the exported document. This is totally a reasonable thing to do! I’m not super familiar with this portion of our code base, but it looks to me like our export operation is not fully synchronous—and closing the document before the operation completes leads to an incomplete rendering of the content in the export. (My first clue that this was happening was that when I created a test document with two identical canvases, and the first exported fine while the second did not.)

When I modify your JXA script’s exportDocument() function to comment out the doc.close() instruction, I seem to be getting much better results. Can you confirm whether this helps your exports get better results?

This is obviously a bug, and I consider it a high priority issue since it means your script cannot reliably work with the data from the export operation—and it has no easy way of knowing when that operation is actually complete.

Returning to the original problem statement for JXA scripting:

I believe this is due to the bug I just identified where closing a document can interrupt an export that hasn’t yet completed (and there’s no way to know when it’s completed). For now, the workaround on this is to leave those documents open.

This problem is solved in the current v7.11 test builds.

The issue you’re running into here is that you’re passing OmniGraffle a string rather than a Path object. You need to use a Path so that the sandboxing system knows that your script has granted OmniGraffle access to the file in question. (Pre-opening the file is a workaround because opening the file also lets the sandboxing system know that it should grant OmniGraffle access to that file.)

If you use OmniGraffle.open(Path(file)) (much like you’re already doing in the to parameter to your export command), you’ll find it solves this issue.

Thanks for testing this, that sounds like an entirely reasonable explanation. Since exporting and closing worked flawlessly in my old Python/OSA code, I would assume that either this bug has been introduced recently, or Python is just so much slower that it never closed the document before it was fully exported.

I will test if simply delaying closing the document will resolve this until the bug is fixed.

Also I wish I had known this earlier, it would have saved me a lot of time in the last month I spent manually exporting stuff. Why did it take 5 weeks from the time I reported this bug as [OG #2166190], complete with the code and the test document, for a developer to look at this?

There’s another thing that needs to be addressed: if you want to get feedback from your customers, allow them to send you files: I tried to sent an email to support, complete with code, test documents and other files to follow up on my tests, but it bounced saying “For security reasons, Gmail does not allow you to use this type of file as it violates Google policy for executables and archives.” The forum also does not allow me to upload those files.

Returning to the problem statement for OmniJS:

This is solved in the current test builds of OmniGraffle 7.11.

When I build a local debugging build of OmniGraffle, I’m able to connect to its JSContext from Safari and use its debugger. However, this doesn’t work for release builds. (This isn’t unique to OmniGraffle: I see the same behavior in other sandboxed apps like https://github.com/kasper/phoenix.git. If someone knows of a solution, I’d be happy to explore it.)

OmniGraffle autodetects changes to plugins.

It might be simpler to invoke your JavaScript directly from JXA, like this:

https://discourse-test.omnigroup.com/t/a-hello-world-document-testing-omnijs-code-from-jxa-text-editor/30817?u=kcase

You might not need plugins at all, but if you do want to work with plugins it’s easiest if you can structure things to let you edit them in place without having to do some sort of publish cycle.

Symlinks can work for this, but because of sandboxing they won’t grant OmniGraffle access to any locations which it can’t already access. You can grant OmniGraffle access to additional locations by going into the Resource Browser and clicking on “Add an External Linked Folder” button at the bottom:

image

I don’t know that I agree with every one of Douglas Crockford’s recommendations, but I think the recommendations made by JSLint are a good starting point. But do listen to his warning:

JSLint will hurt your feelings. Side effects may include headache, irritability, dizziness, snarkiness, stomach pain, defensiveness, dry mouth, cleaner code, and a reduced error rate.

This is a big limitation which has not yet been solved in OmniJS for OmniGraffle. (It has been solved for OmniOutliner, but we made some compromises there which I don’t think we can make in the general case for OmniGraffle. But then again, those compromises are probably better than not being able to work with formatted text at all!)

If you need to work with formatted text from a script, your best bet for now would be to use AppleScript or JXA.

I’m very sorry for the long delay in getting back to you on this bug. I’ve talked with our support team about this, and these sorts of issues should get escalated to engineering much more quickly in the future.

It sounds like you’re sending from Gmail and they’re blocking it? Gmail has nothing to do with our incoming email, and customers email file attachments to omnigraffle@omnigroup.com all the time (including zip files, OmniGraffle documents, scripts, etc.).

Sorry, that must a default setting on this forum software. I’ll investigate and see if we can get that sorted out.

… or delay closing the file ;-)

  1. Some objects in the document can’t be accessed through JXA, again it’s all items in shared layers, but also some groups, tables and subgraphs

This problem is solved in the current v7.11 test builds.

Ok, I tested this already for OmniJS, will have to test for JXA.

  1. Sandboxing sometimes gets in the way of scripting, because OmniGraffle is not allowed to access documents. So I needs to manually open all files before running a script, which is really annoying.

The issue you’re running into here is that you’re passing OmniGraffle a string rather than a Path object. You need to use a Path so that the sandboxing system knows that your script has granted OmniGraffle access to the file in question. (Pre-opening the file is a workaround because opening the file also lets the sandboxing system know that it should grant OmniGraffle access to that file.)

If you use OmniGraffle.open(Path(file)) (much like you’re already doing in the to parameter to your export command), you’ll find it solves this issue.

Ah, that’s really good to know. Its there any documentation that mentions this?

[quote=“kcase, post:23, topic:46128, full:true”]
I’m very sorry for the long delay in getting back to you on this bug. I’ve talked with our support team about this, and these sorts of issues should get escalated to engineering much more quickly in the future.

[quote]

cool

There’s another thing that needs to be addressed: if you want to get feedback from your customers, allow them to send you files: I tried to sent an email to support, complete with code, test documents and other files to follow up on my tests, but it bounced saying “For security reasons, Gmail does not allow you to use this type of file as it violates Google policy for executables and archives.”

It sounds like you’re sending from Gmail and they’re blocking it? Gmail has nothing to do with our incoming email, and customers email file attachments to omnigraffle@omnigroup.com all the time (including zip files, OmniGraffle documents, scripts, etc.).

It was indeed my google account that blocked this. I tested a bit, here’s what I found, because Omni will most likely run into this again:

Google appears to block any ZIP file with a JavaScript file in it. I tested a couple of Plugins, and also some other JavaScript files (e.g. reveal.js)

I guess that means nobody with a google account would be able to send you Omni-Plugins. Your support staff should be aware of that.

BTW, while testing this, I found that my email client (Airmail) automatically zips a plugin when I drag it into the client, so some people might even send a zip without their knowledge.

The forum also does not allow me to upload those files.

Sorry, that must a default setting on this forum software. I’ll investigate and see if we can get that sorted out.

Thanks.

Apple’s documentation for JXA is a bit thin, but the release notes which introduce JXA do briefly mention this (though they just say “you will need”, they don’t describe why):

https://developer.apple.com/library/archive/releasenotes/InterapplicationCommunication/RN-JavaScriptForAutomation/Articles/OSX10-10.html#//apple_ref/doc/uid/TP40014508-CH109-SW31

Paths

When you need to interact with files, such as a document in TextEdit, you will need a path object, not just a string with a path in it. You can use the Path constructor to instantiate paths.

To get to the “why” background, you’d probably have to go back to the release notes for Lion (which introduced sandboxing) or Mountain Lion (which fixed a bunch of the scripting issues introduced by sandboxing).

Thanks, that’s good to know! I’ll remind our support team that whenever customers have trouble emailing us a file (of any type or size), they can use https://upload.omnigroup.com/ to get those attachments to us instead.

I’ve updated our forum configuration to allow more file types. We’ll watch to see if that results in an increase in uploaded spam (which is, I believe, the reason for the default limits).

This export bug should be fixed in r330884 or later. Thank you for alerting us to the problem! (A test build with this fix should be posted within the next hour or so.)

Oh, until now I thought that was pretty much a standard feature of JavaScript Core. For details I can only refer you to some people who made it work: the guys at Bohemian Coding, who explain how to attach the Web Inspector to Sketch for debugging plugins, or maybe Jesse Grosjean of Hogbay Software, because I can use the Web Inspector for debugging Taskpaper Plugins (he even forgot a debugger statement in the code one time, which annoyed some users).

  1. Would I need to open/close omnigraffle whenever I change code in a plugin, or would it autodetect new or changed actions ?

OmniGraffle autodetects changes to plugins.

When I write the OmniJs API test plugin I tried that, it appeared to me this might not work entirely reliably. After a while I switched to manually reinstalling the plugin after each change. That helped.

  1. For my toolkit to process a number of OmniGraffle documents, it appears I would require a plugin, and trigger actions inside the plugin through Script Links (from a command line script or through a service), is that correct?

It might be simpler to invoke your JavaScript directly from JXA, like this:

https://discourse-test.omnigroup.com/t/a-hello-world-document-testing-omnijs-code-from-jxa-text-editor/30817?u=kcase

Am I totally confused now, because I think that is exactly what the official documentation calls a Script Link?

  1. What is the best setup for developing and testing plugins? I can imagine a symlink inside the OmniGraffle plugins folder that points to a working copy, I could also clone the repo directly to the plugins folder (which would impose a certain structure on the repo), or I could “publish” to the plugins folder (e.g. with a makefile) whenever I change some code.

You might not need plugins at all, but if you do want to work with plugins it’s easiest if you can structure things to let you edit them in place without having to do some sort of publish cycle.

Symlinks can work for this, but because of sandboxing they won’t grant OmniGraffle access to any locations which it can’t already access. You can grant OmniGraffle access to additional locations by going into the Resource Browser and clicking on “Add an External Linked Folder” button at the bottom:

image

Oh, that is really helpful information. There’s still the bit where OmniGraffle appears to not reliably reload the plugin code (mentioned above), but I’ll test that again when I find the time to figure out what I did wrong.

  1. is there a specific coding style encouraged for plugins (the example code is a bit inconclusive here)?

I don’t know that I agree with every one of Douglas Crockford’s recommendations, but I think the recommendations made by JSLint are a good starting point. But do listen to his warning:

JSLint will hurt your feelings. Side effects may include headache, irritability, dizziness, snarkiness, stomach pain, defensiveness, dry mouth, cleaner code, and a reduced error rate.

Actually, the first thing I did when I started playing with JXA and OmniJS was set up ESLint. You can see the linter config in each of my repos (.eslintrc). I find linters an essential tool for any project, at least in the CI system. Don’t you use linters at Omni?

Also, when I get some text that is formatted, this formatting is lost.

This is a big limitation which has not yet been solved in OmniJS for OmniGraffle. (It has been solved for OmniOutliner, but we made some compromises there which I don’t think we can make in the general case for OmniGraffle. But then again, those compromises are probably better than not being able to work with formatted text at all!)

If you need to work with formatted text from a script, your best bet for now would be to use AppleScript or JXA.

I’m currently playing with extracting text from the XML directly.

I do understand that people want fine grained control over style of their text, but there is also a usecase for semantic markup, so I think in the long run OmniGraffle should support Markdown, (and maybe even HTML) for text, in addition to RTF, which is a pain for post processing, because there’s zero semantic information in it.

I know that RTF was simple to implement, but proper text (and object) styles would go a long way for working effectively with large collections in OmniGraffle. I would imagine a checkbox in the text inspector that activates Markdown/HTML instead of RTF, and a place to keep a document wide stylesheet. For my usecase - or indeed any translation usecase - that would be a tremendous step forward. Also would solve the problem you mentioned without any compromise.

It appears that those apps are not sandboxed. I can trivially make it work in a non-sandboxed app, too—by simply doing nothing!—but it appears that this is automatically disabled when sandboxed apps are packaged for distribution.

Detecting changes to those files relies on the system’s file coordination APIs. Any well-behaved Mac text editor or synchronization service (e.g. iCloud, OmniPresence) will support this—but it won’t detect files which are updated behind the scenes without using file coordination.

If you keep that file open in Apple’s TextEdit app, does it automatically update with the changes you’re making? If so, that’s an indication that file coordination is happening properly.

Oh, sorry, did I link to the wrong thread? I guess I must have, because that thread has nothing to do with the .evaluateJavascript() call I was meaning to point you towards!

Here’s the post I meant to link:

Will have to check that with the Appstore version of Taskpaper, I just use the regular non-appstore version.

Well, can’t you make a non-sandboxed app available to OmniJS developers? Probably deactivate the plugin update dialog, too?

When I write the OmniJs API test plugin I tried that, it appeared to me this might not work entirely reliably. After a while I switched to manually reinstalling the plugin after each change. That helped.
Detecting changes to those files relies on the system’s file coordination APIs. Any well-behaved Mac text editor or synchronization service (e.g. iCloud, OmniPresence) will support this—but it won’t detect files which are updated behind the scenes without using file coordination.

If you keep that file open in Apple’s TextEdit app, does it automatically update with the changes you’re making? If so, that’s an indication that file coordination is happening properly.

I use Sublime Text, so I think that’s not the problem, but I will test with TextEdit.

Am I totally confused now, because I think that is exactly what the official documentation calls a Script Link ?

Oh, sorry, did I link to the wrong thread? I guess I must have, because that thread has nothing to do with the .evaluateJavascript() call I was meaning to point you towards!

Here’s the post I meant to link:

Using Omnigraffle.evaluateJavascript()

Ah, that makes a lot more sense. Is it just my imagination that it’s interesting to debug OmniJS code that is more than a couple of lines, because the line number for errors in the console would carry an offset, if there’s line numbers at all.

Does .evaluateJavasScript() allow for call an action, or any other code that lives inside an OmniPlugin or Library?