Wall Street spent the week biding its time before Federal Reserve Chair Janet Yellen made a speech on Friday. Then, when Yellen turned hawkish, stocks slumped into the red with the chances of a September rate hike increasing.

Friday's losses added to the week's decline. Over the past five days, the S&P 500 has fallen 0.68%, the Dow Jones Industrial Average slid 0.85%, and the Nasdaq dropped 0.37%. 

The U.S. economy has improved in recent months to the point where near-crises levels of rates are not justified, Yellen said in her widely anticipated speech at the Economic Policy Symposium in Jackson Hole, Wyo. on Friday. The two-day summer summit brought together prominent central bankers, finance ministers and economists from around the world to discuss economic trends and monetary policy action.

"In light of the continued solid performance of the labor market and our outlook for economic activity and inflation, I believe the case for an increase in the federal funds rate has strengthened in recent months," Yellen said in a speech prepared for delivery to the Jackson Hole summit.

Yellen said she expects "moderate growth" in GDP and further tightening in the labor market over the next few years. She also reiterated that the decision on interest rates will "always depend on the degree to which incoming data continues to confirm the Fed policy committee's outlook."

Fischer backed up Yellen's confidence, noting that the overall trends in the U.S. economy are looking better and that Yellen's address is consistent with a September move. Fischer added that the August jobs report out next Friday will factor into the Fed's decision. The Fed will meet on Sept. 19-20.

The chances of a September rate hike increased to 30% on Friday from 21% a day earlier, according to CME Group fed funds futures. A rate hike in December has better chances at 44%.

Fed members' confidence in the U.S. economy soothed investors' fears over earlier data that showed weaker growth over the second quarter. The U.S. economy grew 1.1% over the second quarter, according to the second estimate of GDP from the Bureau of Economic Analysis. The downward revision from 1.2% was as analysts had expected. Weaker manufacturing activity caused by slower international demand and a stronger U.S. dollar put a cap on U.S. economic growth last quarter. Consumer spending remained strong with estimates raised to 4.4% from 4.2%.

Other economic data out this week also backed up the Fed's new hawkish lean. Orders for long-lasting goods in the U.S. in July out on Thursday showed a 4.4% increase. The better-than-expected improvement signaled a recent downturn in manufacturing activity is coming to a close.

The number of newly constructed homes sold in the U.S. reached 654,000 in July, up 12.4% at a seasonally adjusted rate, according to the Commerce Department. The reading topped economists' forecasts of 581,000. New home sales have risen for five straight months.

Oil ended the week 3% lower as domestic inventories continued to build and hopes over a production freeze between Organization of Petroleum Exporting Countries fizzled. The Organization of the Petroleum Exporting Countries will meet in September.

Morgan Stanley analysts threw doubt on whether an agreement between OPEC would come to pass on Monday. Analysts wrote in a note on Monday that a deal was "highly unlikely" as producers remain reluctant to give up market share.

"The market treats OPEC as the central banker of oil where simple jawboning can move markets and scare off shorts," analysts wrote. "However, we see too many headwinds and logistical challenges to a meaningful deal given past statements by OPEC and its members."

HP Inc. (HPQ - Get Report) was among the final few to report quarterly earnings in the past week. The printer company issued disappointing fourth-quarter forecasts as weak sales of its printers weighed on projections. Notebook sales offset softer demand.

In retail earnings, Best Buy (BBY - Get Report) breezed past earnings estimates and reported an unexpected increase in same-store sales. Sales at stores open for at least 12 months rose 0.8%, far better than an expected decline. Online sales drove momentum with the segment seeing 24% growth over the quarter.

Express (EXPR - Get Report) missed quarterly estimates and issued a disappointing outlook. The company blamed "challenging store traffic" as reason for a poor second quarter. Big Lots (BIG - Get Report) reported weaker comparable-store sales than expected, while quarterly revenue declined.

Guess? (GES - Get Report) exceeded earnings estimates but missed sales forecasts in its recent quarter. The fashion brand earned 14 cents a share in its second quarter, 7 cents higher than expected.

Williams-Sonoma (WSM - Get Report) reduced its full-year forecast, with chief executive officer Laura Alber citing a "more cautious consumer." The home-goods retailer reported a 4.8% decline in comparable-brand sales at Pottery Barn, its largest brand by sales.

The second-quarter earnings season is drawing to a close with more than 98% of S&P 500 companies having reported their earnings. Most companies managed to clear lowered expectations over the quarter -- 71% of companies exceeded earnings estimates over the quarter, according to Thomson Reuters.

Expectations had been conservative heading into the season, with many expecting the headwinds of a stronger U.S. dollar, weaker global growth, a prolonged period of low oil prices, and soft manufacturing activity to continue to impact the top and bottom lines of S&P 500 companies.