[Yes! Fixed in 3.1.3 Test Builds] OmniFocus lost hundreds of tag assignments! New tag assignments also get lost


#1

I spent hours adding tags to my items to make up for the fact that you cannot search for items based on their recurrence. E.g., I made tags called “rpt 2w” “rpt 2mo” etc etc.
After spending all this time, I was doing my GTD project review and guess what? Over 400 tags had been deleted! I tried rebuilding the database and guess what? 500 more disappeared!
I have made a video showing the problem.
This is a serious bug.


#2

You’ve probably already done so, but if not, file a report with support (omnifocus@omnigroup.com) - that gets it into the formal support process


#3

This post was flagged by the community and is temporarily hidden.


#4

Thanks I think I did but no response yet…


#6

Omni guys are clearing a backlog of tickets from the OF3 release and got somewhat behind, so support response is slower than we’ve come to expect over the years.

There’s an updated status post here:

https://discourse.omnigroup.com/t/omnifocus-3-tech-support-status-updates/41952/6


#7

Thanks for the update and for staying in communication


#9

Thank you for reporting this bug! We consider this an urgent issue, and now that we’ve figured out how to reproduce the problem we’re planning a new bug fix release (v3.1.3) just to fix it. We hope to put v3.1.3 into public test later this week.


#10

@wealthychef, we think we’ve fixed this bug in the current OmniFocus for Mac test builds:

https://omnistaging.omnigroup.com/omnifocus/

In the release notes for the 3.1.3 test builds you’ll also find a link to access TestFlight builds on iOS as well.

Sorry for the delay in getting this to you!


Upgrade Process
#11

Fantastic quick work. I will check in a week and confirm but it sounds super promising!


#12

Hi, I don’t think that the build I downloaded has fixed this bug.
I can reproduce with a certain item that if I tag it, then rebuild the database, it loses its tag every time.
See this video of the bug in action: https://youtu.be/5pM9Ch_BMlY


#13

Thanks @wealthychef and sorry for the continued trouble! We’ve tracked down another potential source of this behavior and are in the process of fixing it. Apologies for the false start…some scenarios were addressed with the earlier 3.1.3 test builds, but it appears other paths remain. If you can, please email an anonymized copy of your database (Help ‣ Anonymize Database…) to omnifocus@omnigroup.com so we can ensure your case is addressed with the upcoming fixes. Thanks!


#14

OK I have sent you a database.


#15

Omnifocus gave me a script which deleted and replaced every tag in my database, and that combined with using the latest beta release seems to have fixed the problem for me, at least for a half an hour. I’ll check back in a week to confirm it’s fixed, but it is looking promising. Kudos to the Omnigroup team for having an excellent user support staff and working to fix this bug. However, minus 3 points for letting this kind of bug through in the first place; deleting user data is not a good thing! I hope there is now a regression test in the QA process for this… Cheers to all!


#16

BTW, I changed the title to “sort of fixed” because you need both the beta release AND the “replace tags” script to fix this problem if you are having it. Simply running the beta will not fix the problem.


#17

We’re very sorry that we missed this test case as well! Now that we understand the scenario that triggered this problem, we’ve built some automated tests to catch it—and are continuing to work on making the app heal its database without needing assistance from external scripts.


Some background, for those interested:

Our tests of syncing a single version of OmniFocus have always been quite thorough, and if you were always syncing v3 with v3 you wouldn’t encounter this issue. Neither would you encounter this when syncing v2 with v2. We also had very thorough tests of migrating data, so if you migrated a database from v2 to v3 and then using v3 exclusively you wouldn’t have any trouble. Or if you were to sync a mix of v2 and v3 devices for a long period of time (as I did this summer) but never used more than one tag on a task (since they wouldn’t be understood by all your devices yet).

So the reason this bug didn’t surface widely in our earlier testing was because our automated tests were testing the behavior of a single build, and our human testers were generally exclusively syncing v3 devices—or avoiding the use of multiple tags until they were able to upgrade all their devices to v3, at which point there was no longer an issue.

Oh, and there was one more complicating factor: when people were syncing v2 with v3 and using multiple tags, things appeared to work just fine when you made individual changes and synced them between the two. The problem only appeared when v3 was rebuilding its local database (which is usually only done right after you install a new build of the app).

I haven’t fully dug into all the edge cases, but the basic problem was that v2 didn’t understanding those extra tags being written by v3. So, for example, one of test cases we missed was this:

  • In v2, create an item with context ‘C’ and sync to 3.1.3 (here, r320529)
  • Add a second tag ‘Tag X’ to the item and sync
  • Back in v2, change the context from ‘C’ to ‘D’ and sync
    → v3 shows both tags in UI — so far so good!
  • In v3, File ‣ Rebuild Database…

Result:

  • ‘Tag X’ falls off and item only has ‘D’ assigned to it

Expected:

  • ‘Tag X’ remains as second tag

Because v2 doesn’t know anything about tag relationships, it was simply writing a new “context” assignment for the edited item. But tags have a much more complex relationship with items than a simple assignment; they maintain a database join table with entries that represent the many-to-many ordered relationships between tags and items. So when v3 reads a v2 edit which edits an item’s context, it infers that it make several changes to the join table entries between tags and items:

 * Insert a join table entry relating the item and the newly set context D
 * Remove the join table entry relating the item and the previous context C
 * Re-rank join table entries on the item to make this tag primary, starting with rank .zero

It turned out that the task would only lose its extra “Tag X” if the ranks on the join table entries were originally before rank .zero (i.e. some were “negative”).

The underlying bug turned out to be that v3 would write the join table entry insertion and removal, but not the update that adjusts the rank of the entry relating the task and Tag X. If this rank was originally before .zero, a rebuild would then place Tag X at the start of the ordered list, meaning that the task’s ordered tags were [X, D] instead of [D, X]; the usual context fix-up logic would then move D to the first position, removing X.


And that’s where things stand right now. Until this bug is resolved, I would recommend not using multiple tags in v3 until you’ve fully migrated away from using v2. (Unless you’re OK with the risk of losing the extra tags on an item when you change its context using v2 and rebuild your database.)


#18

This evening we posted a new test build of v3.1.3, which hopefully fixes this remaining edge case. (It should also automatically repair databases that had gotten into a corrupt state, which unfortunately means those databases will lose some tag assignments when you first update—but this repair should prevent those not-properly-recorded assignments from getting lost at some mysterious point in the future.)

  • Tags — Fixed another problem that could cause lost tag assignments after an item’s context was edited in OmniFocus 2. Future changes to contexts in OmniFocus 2 should now always replace the first tag in OmniFocus 3, leaving other tags unchanged. However, you may see some tag assignments disappear after updating to this build —we’re very sorry for the inconvenience.

Safe to update to OF 3? And who is still in OF2?
#19

I have revisited this and it still looks FIXED to me, after running the script once. I don’t have to do anything and zero items have lost tags since last week. I’m using the latest beta referenced in the above post by Omni Staff.