Paper Size

I'm not sure precisely what those dimensions are, but you can, via Layout > Page Settings.

The paper sizes available under Layout>Page Settings include everything from A4 and Letter all the way up to Folio and Tabloid, so the limitation is really not what MuseScore can handle, but what your printer can handle. Very few desktop printers can handle paper larger than 11x17 (known as 'Ledger'). If you want to use the older professional publishing standard page of 9x12, you will need to invest about $8000 in a commercial laser printer which can handle 12x18 sheets.

It is an unfortunate fact that the hardware available in any given era tends to define the print product produced. POD micro-publishers are generally limited to 8.5x11 page size because the largest sheet most inexpensive printers can handle is 11x17. As a result, in today's publishing world there are only a few, very high-end publishers which still print music on 9x12 pages, and not all of them print all their editions on that size, either. Barenreiter, for example, uses 8.5x11 for its lower-priced line of reprint editions; their top-drawer Urtext stuff is all printed on 9x12...but you pay for it (of course!).

As a work-around, you can set MuseScore to produce a PDF for 9x12 pages, and save the file to a thumb drive. Take that to your local copy shop and they should be able to print it for you in booklet form.