Bush allowed three runs and five hits, pitching into the ninth inning to improve to 3-and-oh in his career against Houston. He struck out seven and walked one.
Bush came out for the ninth inning but couldn't record an out, allowing Lance Berkman's R-B-I double before he was replaced by Francisco Cordero.
Morgan Ensberg hit a run-scoring single with one out and Adam Everett walked with the bases loaded to cut it to one.
But Cordero recorded the final two outs for his seventh save in seven chances this season.
The first-place Brewers have won four of five and are now 11-and-7 this season.