Wall Street made its best effort to extend a three-day winning streak into a fourth session on Thursday but last-minute moves toward the flatline weren't enough.

Stocks were well off session lows, but still closed in the red after a pullback in crude prices pressured the energy sector. The S&P 500 was down 0.17%, the Dow Jones Industrial Average fell 0.11%, and the Nasdaq declined 0.32%. Losses were more than double earlier in the day.

The S&P 500 has enjoyed three straight days of gains, pushing it within 1% of its all-time high. The benchmark index has been helped along by a rally in crude oil and a less-hawkish Federal Reserve, which all but eliminated the chances of a rate hike in June.

Crude oil fell below $51 a barrel Thursday after scoring its best settlement since last July Wednesday. The commodity was on a tear Wednesday after the latest weekly data showed another decline in U.S. inventories. A general downward trend in U.S. production, which has eaten into crude supplies, has recently buoyed prices.

West Texas Intermediate crude oil fell 1.4% to $50.50 a barrel on Thursday.

"The petroleum markets are seeing light-volume profit-taking in Thursday trade as the U.S. dollar index firmed by 0.5% on lower-than-expected U.S. unemployment claims," said Timothy Evans, energy futures analyst at Citi. "Although there is potential for a small decline to develop into a more significant drop, so far this looks like a modest technical correction following three days of gains rather than a major reversal."

The number of new claims for unemployment benefits in the U.S. continued to decline in the past week, falling 4,000 to 264,000 in the first week in June. The less-volatile, four-week average dropped 7,500 to 269,500. The declines are a positive after a surprisingly weak May jobs report, which raised concerns over the health of the labor market.

Restoration Hardware (RH - Get Report) tanked after slashing its quarterly and full-year outlooks. The furniture retailer warned profit could tumble by as much as 40% this fiscal year as it tries to cut back on the number of discount offers.

JM Smucker (SJM - Get Report) moved higher, leading the S&P 500, after topping analysts' estimates on its top and bottom lines in its recent quarter. The owner of Folgers coffee reported a 25% increase in net sales in its fourth quarter thanks to growth in its retail coffee segment in the U.S. Retail coffee revenue rose 9% over the quarter.

British telecom Vodafone (VOD) fell more than 2% after agreeing to combine its New Zealand operations with the country's unit of Sky Network. The $2.4 billion merger will combine Vodafone's 2.35 million mobile connections in New Zealand with Sky's roughly 830,000 subscribers.

Medical device maker Boston Scientific (BSX - Get Report) announced a global restructuring program which will save around $115 million to $150 million annually by 2020. Boston Scientific said it expects "some employee attrition and targeted headcount reductions" under the restructuring, but anticipates that its overall employee base will remain relatively unchanged.