Q. 4: David tried a handful of desserts from the table, which ultimately gave him an upset stomach.
The answer key ->
A. David tried a handful of the table's desserts, which ultimately gave him an upset stomach.
Is it ok to use "that" here instead of which and remove the comma:
David tried a handful of the table's desserts that ultimately gave him an upset stomach?
The distinction you're drawing is essential vs. non-essential modifiers. If you use "comma+which," the modifier is additional information that is not necessary to identify the modified noun.
"That" makes it an essential modifier -- that is, the information contained in the modifier is necessary to describe precisely the nouns you mean, perhaps to avoid confusion.
Does the second sentence work? Well, not really. It implies that David actively sought the desserts that would ultimately give him an upset stomach, and that he chose these over choosing desserts that would not give him an upset stomach. Also, David didn't try just a handful of the desserts that would ultimately give him an upset stomach -- he ate all of the desserts that would give him an upset stomach. This sentence clearly conveys a different, nonsensical meaning.