I have long since concluded that, in films in TV, people can only hear things if the camera is pointing at them. The people who live in movies clearly understand this, since people engaged in plotting frequently walk 3 steps away from the Person Who Must Never Find Out in order to talk about their plot, secure in the knowledge that they won't be overheard So maybe characters in books place their faith in a literary version of this, and assume that if the Person Who Must Never Find Out doesn't appear in this chapter, there's no possibility of them every reading their words.