As you say, American English uses *yard* in much the same way that British English uses *garden*. The prototypical American house is situated on a plot of land in such a way as to effectively separate the plot into a *front yard* and a *backyard*. (Why the latter is a compound word and the former isn’t is a mystery for the ages.) The front yard is the more publicly facing of the two. If the plot is wide enough, there may also be a *side yard* (or two). 

Anyway, Lee’s use of the word *yard* here, with no other words or phrases to modify it, definitely implies the existence—on the plot of land opposite them—of a house.