Learning to Listen with Discernment