How are libraries legal?

I was just wondering how libraries are considered legal. I borrow audio CDs all the time, and I've ripped above 60. How is this legal? This takes money from the artists just as much as piracy does, but somehow it's perfectly fine. I mean, even with books, authors would make so much more profit from what they do. Even if it is legal, how is this even slightly ethical?