Hi,
Welcome to the forum. If you're trying to be cost-conscious (and who isn't these days?) then your best bet is to buy the BD55 and try the analog 5.1-channel hook-ups to your current receiver.
There are a few reasons why this is superior to just using optical/coax digital connections, perhaps the most important of which is that fiber/coax connections cannot support discrete multi-channel PCM soundtracks, but analog multi-channel connections can. Many Blu-ray Discs use multi-channel PCM soundtracks, and if you try to pass these over fiber/coax, you'll get a down-mixed 2-channel version instead of the multi-channel mix. Also, by using the multi-channel analog outputs of the BD55, you can take advantage of the new lossless surround sound formats - DTS-HD Master Audio and Dolby TrueHD. The BD55 will convert these internally to multi-channel analog output without having to revert to the lossy soundtracks (regular DTS or Dolby Digital).
Some people who use this approach actually connect the fiber or coax output between player and receiver *also* so that they can use this for regular DVD playback. The BD35 and BD55's support for DTS-HD is called "DTS-HD Master Audio Essential" and this means the internal DTS decoder does not support the DTS legacy "extensions" (DTS 96/24, DTS-ES 6.1), so the best way to support legacy DTS soundtracks from DVD is via a digital connection to your receiver (which in your case means fiber or coax). The other benefit to connecting things this way is that you can take advantage of your receiver's built-in bass management via the digital inputs (again, when watching regular DVDs).
In terms of your other question, HDMI audio is the best overall, because it can pass all of the new formats (Multi-channel PCM, Dolby TrueHD, Dolby Digital Plus, DTS-HD High Resolution Audio, DTS-HD Master Audio) digitally to a compatible receiver. And, because it is carried digitally, your receiver's bass management, EQ and delay settings will all be respected. This can make set-up a lot simpler. If you use the multi-channel analog outputs instead, then you will have to do level, bass management and delay settings in the player itself, which is a bit more limited.
And as for the difference between fiber and coax, in my opinion, there is no difference. Format-wise, the underlying data stream is identical between both, so if you get the signal delivered to your receiver properly, there should be no audible difference between coax and fiberoptic connections.
So, again, my advice is to buy the BD55 and 6 analog RCA cables and try things out via the hybrid multi-channel analog/fiber optic or coax digital approach. If you like what you're hearing, then it will save you the expense of a receiver upgrade. The BD35 is actually getting a little hard to find these days (possibly temporarily), but stock on the BD55 is still pretty good so you can get it for a decent discount below MSRP. You can support the site by buying it via the below link:
DMP-BD55 on Amazon.com
Hope that helps.
-Chris