I think the main reason that I don't like telephones is that phoning somebody is a very intrusive way to contact them - you're interrupting whatever they are doing to effectively say "talk to me now - I am more important than whatever you are doing". Similarly, I dislike being phoned in general because it interrupts whatever I'm doing.
When I do have to phone someone I always try to make sure I have all the information I need in front of me and have answers to any questions they might ask, so as not to waste their time any more than is necessary.
Email is so much better because people can check it when they like and answer when it is convenient for them to do so. I also prefer replying to emails because it gives me a chance to think about what I want to say. I prefer not to do business with companies which will only communicate over the phone, when I get the choice (which doesn't seem to be very often).
Another reason is that I sometimes find it difficult to understand what people are saying on the phone, especially if I am unfamiliar with their accent. Not only can you not lip read to give you that bit of extra disambiguation data, but you also don't get to see the body language which tells you so much about what a person really means.