Stocks extended their decline on Wednesday afternoon as crude oil closed at its lowest settlement in roughly a week. 

The S&P 500 was 0.36% lower, the Dow Jones Industrial Average fell 0.25%, and the Nasdaq slid 0.56%.

Crude oil pulled lower after a weekly reading on U.S. oil inventories showed a build. The Energy Information Administration showed an increase of 2.5 million barrels of crude to stockpiles over the past week. Analysts had expected an increase of just 200,000 barrels.

West Texas Intermediate crude oil fell 2.8% to $46.77 a barrel on Wednesday, hovering at a one-week low.

Sales of previously owned homes in the U.S. fell 3.2% in July to a seasonally adjusted annual rate of 5.39 million, according to the National Association of Realtors. Tight inventory has choked some of the demand in recent months, leading to higher prices across the board. Economists expected a pace of 5.48 million.

Wall Street has seen jittery trading so far this week as investors opt to sit on the sidelines until the kickoff to the Federal Reserve's annual Monetary Policy Symposium in Jackson Hole, Wyo., on Thursday.

"One issue that is likely to garner considerable attention during the symposium is the outlook for the neutral fed funds rate," Deutsche Bank analysts wrote in a note. "Having fallen to -- and remaining persistently near -- record low levels, the neutral rate has drawn intensifying scrutiny from key Fed officials. Its outlook has pivotal implications for both the likely future path of the fed funds rate and the way in which monetary policy is conducted in the future."

Yellen will cap off the week with a speech at the summit which will provide important context to the markets' outlook for rates. Yellen's comments will likely remain dovish and conservative, pushing data dependency for any future rate moves.

The chances of a September rate hike currently sit at 18%, according to CME Group fed funds futures. A rate hike in November has better chances at 41%.

In earnings news, Express (EXPR - Get Report) tanked 25% after missing quarterly estimates and issuing a disappointing outlook. The apparel retailer earned 13 cents a share over the quarter, down from 25 cents a year earlier and below consensus of 17 cents. Same-store sales tumbled 8%, sharper than an expected 4.6% drop. Current-quarter earnings guidance between 9 cents and 15 cents a share fell well below expectations of 32 cents. The company blamed "challenging store traffic" as reason for a poor second quarter.

Tesla (TSLA - Get Report) moved slightly lower after it revealed of new versions of its electronic vehicles. The company announced new versions of its Model S and Model X vehicles late in the trading day Tuesday, including details of new batteries for faster performance. Expectations had been high after CEO Elon Musk made mention of an announcement earlier in the day.

Pfizer (PFE - Get Report) extended its spending spree on Wednesday, purchasing the commercialization and development rights to AstraZeneca's (AZN - Get Report) late-stage small molecule antibiotics business outside the U.S. in a deal that that could be valued at more than $1.5 billion. The deal is expected to close in the fourth quarter of 2016. Pfizer agreed to purchase cancer specialist Medivation at the beginning of the week.

Fitbit (FIT - Get Report) climbed 2% after a U.S. International Trade Commission judge ruled that it hadn't stolen rival Jawbone's trade secrets. Jawbone had hoped to secure an import ban against Fitbit's wearable fitness trackers. The judge found that "no party has been shown to have misappropriated any trade secret."

Garmin (GRMN - Get Report) was downgraded to sell from neutral at Goldman Sachs on Wednesday. Analysts said fitness segment growth is decelerating. Shares fell 6%.