but there's no way to delete pages.
Ok, that's precisely true but not exactly helpful. Writer only generates pages where the content calls for them. So you can only remove pages indirectly, by removing the content, or changing the settings in the content that call for the extra pages.
There are two things to look for:
1) Empty paragraphs. View > Nonprinting characters: ON and look for any lines that contain only spaces and/or paragraph marks, ¶
Use the delete key(s) to remove them and close up any text around them. With the content more compact, it will use fewer pages.
2) The second thing that can lead to extra pages is page breaks. These are special settings in a pargraph that tell Writer "start a new page right here". These can be harder to find because they're hard to see, but you can look at the first paragraph on a page where the previous page has extra space at the bottom. That's usually an indication that the first paragraph on the new page has a page break. You can check for sure by clicking in the paragraph, then menu Format > Paragraph > Text Flow ... and look for check marks next to Breaks > Insert: ON, Type: page.
Again, as you turn off the page breaks, the text will close up (compact vertically) and thus require fewer pages.