My question now is, is there a special term for a book without a plot?
Both "concept books" (covering concepts such as colors, numbers, words) and "slice of life books" are two possible correct designations of picture books without plots, depending on the book.
When I Was Young in the Mountains, by Cynthia Rylant is a great example of a "slice of life" book.
There is a variation, which in some cases is a cross between a concept book and slice of life, which I designate "slice of time" books: books that deal with seasons, the day, or other aspects of time.
There are "also cumulative books (which don't always have a plot) and these have been around for centuries. The accumulation can also be created with the illustrations.
There is another category which I call "cause and effect" books. (Remy Charlip's
Fortunately is an excellent example.)
It is common for picture books to use more than one category in the text.
I did a lot of research into atypical narrative arcs in picture books during my picture book semester of my MFA program (at Vermont College). I found 11 significant categories of picture books without plot. I have presented my findings at both Vermont College and in other venues, as part of a writing workshop.
Basically, not all books have plots. Awareness of other types of narrative arcs, opens up a world of possibilities for the picture book author.