Babies are naturally curious. It's how they learn. They touch things, they taste things and they play with everything in sight.
Some babies seem to be more curious than others, taking a long time examining each and everything they can get their little hands on. I would have to say my third child was my most curious baby. Even though he was a very content baby, he was just into everything. To this day, he is the most adventurous of my children.
Keep your babies safe and watch what they get hold of, but by all means, let them explore the world around them.