Device drivers are difficult to write. Understanding the hardware can be the hardest part. Often the documentation is hard to read and understand because it is written from a hardware designer's perspective (that is, if you can get the documentation -- many manufacturers are very reluctant to give it out). Drivers work directly with the kernel - a bad pointer can crash the whole OS. And, finally, you can't use warm and comfortable debugging tools, since you are working in the kernel.
