We stayed at Hyland House for three nights. It's a great location, right near the Botanic Garden (a five minute walk) and next to Woodhaugh Gardens (nice trail to Ross Reservoir). The hotel is a 15 minute walk to downtown. We'd stay here again; the rooms were nicely appointed and had great bathrooms and great kitchenette facilities (dishes, silverware, toaster, hot water pot, fridge, microwave - all common in Kiwi motels). Free laundry, friendly hosts. The only drawback is that Hyland House is on one of the main streets (as most of the motels are). So, bring earplugs. We closed the heavy curtains which definitely helped. Also, the windows are triple-glazed.
Dunedin is a great university town - the railway station is beautiful and the Robbie Burns pub is fun. We took a day trip to Oamaru to see the blue penguin colony at night. Then, we took one of the best trips we've taken in NZ - a daytrip to the Otago Peninsula with Twilight Wildlife tour - small groups and great tour guides! We saw albatross flying at the Royal Albatross Colony and yellow-eyed penguins at Penguin Place. The yellow-eyed are some of the world's rarest penguins and you can see them up close! The Twilight Wildlife tour was worth every penny. The Otago Peninsula is beautiful but the roads are narrow and windy. Taking the tour allowed us to enjoy the views without worrying about the driving.