I have been to both islands. It depends on what you are looking for. The Brac is waaaaay lower key than GC. There are not many restaurants. If you are staying in a house or villa, it will definitely be quiet. We stayed in a house and if I saw someone walking along the beach I hurried outside and would stop them to talk……my husband thought I was nuts….he was happy to have no one around.
The shoreline in the Brac is often iron shore, although there are some beaches, but nothing like Grand Cayman. I think there is more shore snorkeling in GC.
Did you stay on SMB when you were in GC? If you want a lower key experience, but not as low key as the Brac, you might try the other side of GC….northside or Rum Point area.