Heat oven to 350°.
Trim rough ends of asparagus.
Using a vegetable peeler, carefully peel outer layer from the bottom 2 inches of asparagus spears.
Bring a pot of water to a bowl, add asparagus, cook about 2 minutes until bright green.
Rinse asparagus with cold water and reserve.
Melt butter and olive oil in a medium skillet over medium heat.
Add mushrooms, onions and cook until onion is soft and translucent, about 15 minutes.
Remove the pan from heat and set aside.
Randomly arrange half of the bread pieces on bottom of prepared casserole dish.
Sprinkle half of the rosemary over the top.
Place half of the asparagus, onions, mushrooms, and cheese on top.
Make a second layer with the remaining bread, rosemary, asparagus, onions, mushrooms, and cheese.
In a large mixing bowl, combine, milk, yogurt, eggs, salt and pepper and whisk until blended.
Pour egg mixture over contents of the casserole dish.
Press down, completely soaking top layer of bread, and let pudding stand for 20 minutes.
Bake until custard has set and top is golden brown, about 45 minutes.
Remove dish from oven, and transfer to a wire rack to cool.