With only the keyboard, is there a way to navigate to or check/uncheck the “Open in a new window” box in the Quick Open window?
Use a Keyboard Maestro macro similar to the one below. Fire it when the Quick Open palette is open. Very reliable.