For XP and Vista, there a few a different ways reported that allow you to do this. I'll point you in a few different directions, but have not actually tried any myself. I only used the Windows 7 fix.
For both Excel 2003 and 2007, the setting for "Show all Windows in Taskbar" is said to work by some. In Excel 2003 you can find it in Tools->Options->View. In Excel 2007 you can find it by clicking the Windows button in the top right, then going to Excel Options->Advanced. From there scroll down to the Display section and check the box. This solution was posted here
For an XP specific solution, you can supposedly make a change to the file associations to force a new window when opening each spreadsheet. Here's how to do it:
1. Go to My Computer->Tools->Folder Options->File Types, choose XLS, then go to Advanced.
2. Uncheck "browse in same window"
3. In the Actions box, highlight Open and click the Edit button
4. In the "Application used to perform action:" box, remove the /e at the end and instead insert "%1" with the quotes. It should look like this: "C:\Program Files\Microsoft Office\OFFICE11\EXCEL.EXE" "%1"
5. Check the box for "Use DDE"
6. Delete anything that is in the "DDE Message:" box, and leave it blank
7. Make sure "Application:" says Excel
8. Empty the "DDE Application Not Running:" box
9. "Topic:" should say system
10. Click OK twice to get back to the File Types tab
11. Repeat steps 1-10 for the XLSX extension (if it's listed)
For Windows Vista I wasn't able to find a specific fix. However, when I used Vista I had no problem with spreadsheets opening in new windows and do not think I changed anything. If you are having the issue in Vista, I'd suggest first trying to uncheck the "Show all Windows in Taskbar" option named higher in the article. If that doesn't work make sure to go back and recheck that box, and then try using the same registry hack below that fixes Windows 7.
And of course, for Windows 7 you can do the following
1. Open Regedit so you can modify your registry. If you're worried about messing something up make sure to back it up first. You can do that by going to File->Export... and choosing a save location
Hopefully that helps. I know it helped me. Thanks go to Turbo2001rt in the Microsoft TechNet forums for the fix. If you want to see the original TechNet article that helped me, you can check it out here. If you made it all the way through this article and either don't know what an xlsx file extension is for, or do not have xlsx as an option when you change your file type, check out my previous post for info about Microsoft's Office 2007 compatibility pack download.