Is it cheaper to custom build built in bookshelves or customize with various moldings and trim ready made bookshelves?

It would be MUCH cheaper to customize ready-made bookshelves than to have them built to spec, even if you do the work yourself. The manufacturers of ready-made or flat-pack shelving have the advantage: they fabricate hundreds of thousands of board-feet of components every year, quickly and efficiently, with the economy of scale necessary to keep the cost down. The quality of mass-produced items does vary widely, and will never match custom pieces - but do your homework, and select the simplest pieces with the best joinery, and you'll end up with nearly-custom results.