Use that without a comma when you are referring to something defining or limiting that could not be removed from the sentence. Use which with a comma before it when you are referring to something specific that adds information but could be removed from the sentence without losing the sense, e.g.,
- He spoke about the books that they all liked.
- She spoke about old English classics, which was the topic she had been given.
The sentence “He spoke about the books” is incomplete (the article the makes you ask “What books?”), whereas the sentence “She spoke about old English classics” is complete.
Note that, for plain language, you should say, “He spoke about the books they all liked” because the word that is unnecessary in this sentence.