Good books are an excellent way to open mathematical discussions with children. There are quite a number of good picture books that correlate with mathematical topics. Picture books can be enjoyed by more than just preschoolers and kindergartners. My 9-year-old son reads nearly every picture book I check out of the library. My 11-year-old daughter reads the "good ones".
The following is a list by mathematical area of picture books that are fun to read. Each book is listed once, although many could be put in several categories. Most of these books can also be found at your local library.