Heat oven to 425 degrees and place a rack in the lower-middle position.
Cut slits, 1/8-inch to 1/4-inch apart, into the potatoes without slicing through so that the slices stay connected at the bottom of the potato. You can rest the potato in a large serving spoon and use that as a guide for when to stop slicing — slice straight down and when your knife hits the edge of the spoon, stop.
Place potatoes on a baking sheet; brush with olive oil and sprinkle with salt and pepper.
Bake for 30 minutes, then remove from oven and brush again with olive oil. Make sure to get in between the slices. Place garlic slivers and rosemary in the potato slits. Bake for another 30 to 40 minutes, until the potatoes are crispy on the edges and the middles are easily pierced with a paring knife.
Serve hot as a side dish.