This may strike horror into the hearts of the test-driven-development crowd, but it makes me nervous when I write a program and it works correctly the first time. Sure it works, but maybe the code is subtly broken in some way and it only works by accident. Maybe if you sneeze too loudly nearby it will blow up. Maybe if I add some feature it will blow up. Until I've given it a decent workout there's just no way of knowing.
But a program with a bug is a puzzle - solving the puzzle gives one purpose, and an opportunity to learn more about how the code works (or why it doesn't). Sometimes in the process of debugging I'll change parameters or add debugging code, exploring the design space in order to get a better idea about what's going on. Sometimes doing this will give me ideas for improvements unrelated to fixing a particular bug - a way to improve performance or a new feature to add.
I have much greater confidence in a program that I've fixed lots of bugs in than one that works first time.