'What would be the train/bus station to take, and is it easy to get to Hay on Wye by public transport?' Although you could travel by coach from London, taking the train for most of the journey will likely be the better option. 'Will I need to take the bus to one station and then find another bus to Hay on Wye?' You will need a local bus service from Hereford to Hay on Wye, but not from London.
On what date do you plan to travel to Hay on Wye, and will it be a single or return journey? You mention a stay 'in the Hyde Park area' which I suspect means Bayswater but could be Paddington; where specifically is your start point? You may take a GWR train from London Paddington to Hereford and a local bus from there to Hay on Wye, with the bus stop immediately in front of the rail station in Hereford.
Let us know more, please.
There is a direct bus service from Hereford to Hay, approximately every 2 hours (except Sundays, which has a different service). Unfortunately the direct trains from London to Hereford arrive a few minutes after the bus departure. A shorter overall travel time is achieved by changing at Newport. Use the Travelplanner below to check on journeys, and compare train times on the National Rail web site. You will need to make a choice between kicking your heels in Hereford for over an hour or a simple change of train at Newport to reduce travelling time. The 11:30 National Express coach co-incides nicely with 15:55 Bus from Hereford, the equivalent train journey would commence at 12:15.
This is a trip I make frequently and would advise hiring a car once you get to Hereford to drive the rest of the way to Hay.
Don't know how long you're staying but there is lovely countryside around Hay including Hay Bluff as well as villages and country pubs in the area.
Exploring by car is fun and enjoyable.