Omni Sync Server flooded with old data?


#1

Hi,

What happens to old action items on the Omni Sync Server when I delete them via one of the clients? Do they stay around in some sort of hidden “flagged as deleted tombstone” mode and slow down syncing, or do they get fully deleted from the omni sync server database as if they never existed?

Many other syncing programs use tombstones (like 1Password) to let all devices know that an item has been deleted. And I worry that Omni Sync works the same way and that deleted items stay on the server and bog it down with “deleted” data.

This worry leads to me making less separate actions than I’d like to.

I’ve heard that the sync server is actually just a bunch of zip files with a chain of atomic add/modify/delete changes in each zip, and that the chain is condensed every time all devices are at the same point on the chain. In that case, I guess you’re not using tombstones, and that it’s safe for me to flood the database with 1000 items and then delete them again, without ever worrying that deleted items are going to slow down future syncing?


#2

In 2014, @tekl said that deleted items are completely removed from the database.

Since the WebDAV sync just seems to be “normal OmniFocus database + list of zip files with additions/edits/deletions for that database”, I guess that answers my question… deleted items are removed from the server when all devices have synced past the time of deletion and consolidated the base-database on the server.

So it seems we can create as many actions as we want and never worry about database bloat over time. Well, that’s liberating. Wow!

Please let me know if I got this wrong.


#3

I think you’ve got it right here, as long as you remember the distinction between Completed and Deleted. If you’re using OmniFocus for Mac, you can use the Archive function to remove completed items from your syncing database that you don’t want to delete, but don’t need to have access to on your other devices.


#4

@dave Thanks for the awesome tip! I had no idea about the archiving feature. I’ve been pressing the delete key to “complete” my tasks. I’ll now start using the normal completion checkbox, and just periodically use Archive to flush out the completed items. Much easier and less nerve-wracking (more than once I’ve held down delete a bit too long and removed multiple items).

The reason I started worrying about all this is that sync has always been kinda slow, so I tried to keep a light database. And recently it’s been taking 2+ minutes every time, making me think the years of old deleted actions were the culprit. But I found the cause thanks to this forum… I had an old OmniFocus installation still associated as a sync client, and it was bogging down everything since every client had to maintain a long zip file chain of changes just in case the old client ever came back. I deleted the old client from the sync preferences, which made all the modern clients consolidate the zip chain and merge the base database to a more recent common point, and now syncing is pretty fast again (less than 20 seconds).

I’m very happy to have learned that I can spam the database with separate actions without slowing down the sync over time. ;-)

Edit: I found another cool way to delete completed actions visually:


#5

In the interest of completeness, I found an old OmniFocus 1 topic from 2011. OF1 also synced using WebDAV and I think the technology is pretty much still the same, so this extra information may be relevant today too (especially for those who don’t sync with servers):

Actually, pressing delete just adds a transaction to the database saying “this item has been deleted”, similar to the ones saying “this item has been completed” or “this item has a due date of May 7, 2011, 3:37 PM”. It doesn’t actually delete anything until your database is compacted. Until that point, one could still recover a deleted item, and if you were deleting something to get rid of its footprint (a large attachment, for example), it won’t actually reclaim the space.

If you are not syncing OmniFocus to another iOS device (or Mac), database compaction only happens when you request it with File->Compact Database. If you are syncing, the Compact Database menu item will be grayed out, and compaction will happen automatically when all of your devices have reached the same sync point and there are transactions more than an hour old (transactions less than an hour old are not compacted).


#6

I came back to explain this, but it looks like you found it!