There are a few minor things getting in the way of your desired results. First is the quartz filter. I looked into how it works, and it reduces file size by shrinking your PDF without regard to the image quality and it sets the page size maximum to 512 pixels, so you want to discontinue use of that for any purpose other than a preview. It will make your graphic too low a quality to print in most cases. If the PDF files being too large are an issue, consider using some kind of cloud provider like iCloud or Dropbox to send your larger documents to the printer.
The second issue is with OmniGraffle and the way pages are defined. Because it sounds like you are printing with bleeds, you likely want to use a fixed page size instead of letting page settings that may have printer margins define your page size. Here is an example document that has settings you could use: 7by9Portrait.graffle (19.1 KB). Importantly, the units have a * at the end, so are one of the scales that prints true to size at 100% page scale, as shown in this screenshot.
When you check the size of the graphics in the Geometry Inspector, and then you export to PDF at 100%, you get a 7" by 9" inches actual size image. You can check that in Preview by going to Tools>Show Inspector. There the page size should be showing at 7" by 9". Check the page size in Preview before running the filter on the PDF, and only run the filter on a copy of the PDF so that you can compare. If the page size is different after running the filter, you can be certain that is the cause of the size difference.
If you are still running into any inconsistency, feel free to send us an example document from Contact Omni in the help menu. We’d be glad to work with you to make sure you get the right results.