A massive rally in crude oil on Friday wasn't enough wasn't enough for Wall Street to close on a high note.

Benchmark indexes pared earlier gains by the final hour of the session as heavy losses in the retail and consumer sector overshadowed highs in energy. The S&P 500 was up 0.28%, the Dow Jones Industrial Average added 0.2%, and the Nasdaq rose 0.05%.

Consumer stocks were the weakest performers on Friday, led by a selloff in Gap (GPS - Get Report) shares. The clothing retailer plummeted 14% after reporting a 6% slump in same-store sales in March. The retailer had reported a 2% increase over the same period a year earlier; its Banana Republic stores were the worst performers, tumbling 14% compared with a 3% decrease in March last year.

Other retailers in the red on Friday included Nike (NKE - Get Report) , Target (TGT - Get Report) , VF Corp (VFC - Get Report) , Dollar General (DG - Get Report) , L Brands (LB - Get Report) and Ross Stores (ROST - Get Report) . The SPDR S&P Retail ETF (XRT - Get Report) fell 0.5%.

Target is a holding in Jim Cramer's Action Alerts PLUS Charitable Trust Portfolio. Want to be alerted before Cramer buys or sells TGT? Learn more now.

Crude oil enjoyed another major rally on Friday after the number of active oil drilling rigs in the U.S. fell by 8 to 354, according to Baker Hughes data. The number has tumbled 78% since hitting a peak in October 2014.

Oil was already higher as commodity traders pinned hopes on next week's meeting of the Organization of Petroleum Exporting Countries, which may lead to a production freeze. West Texas Intermediate crude oil closed 6.6% higher at $39.72 a barrel.

"Sentiment is ... swinging back to a more bullish view of the upcoming April 17 producer freeze talks in Doha," said Timothy Evans, energy futures analyst at Citi. "We continue to see any deal without Iran as doing little more than confirming a status quo, since the majority of producers in attendance weren't planning or able to raise output anyhow."

The energy sector was the best performer Friday. Major oilers including Exxon Mobil (XOM - Get Report) , Chevron (CVX - Get Report) , PetroChina (PTR - Get Report) and Royal Dutch Shell (RDS.A - Get Report) were higher, while the Energy Select Sector SPDR ETF (XLE - Get Report) added 2%.

Marriott (MAR - Get Report) and Starwood Hotels (HOT) were slightly higher after shareholders approved their proposed merger. A near-unanimous 97% of Marriott shareholders approved of the acquisition, while 95% of Starwood voters were in agreement.

Stocks have had a rocky week, rallying on Wednesday on signs of a dovish Federal Reserve and selling off on Thursday alongside crude oil. Investors have shown wariness ahead of the first-quarter earnings reports, which begin next week. The Dow suffered its worst loss in six weeks on Thursday, plummeting 174 points or 1%.

Fed Chair Janet Yellen and former Fed chiefs Alan Greenspan, Ben Bernanke and Paul Volcker inspired good vibes on Wall Street, however, with assurances that the U.S. economy isn't heading back into a recession despite presidential frontrunner Donald Trump's claims of a bubble.

The "economy has made tremendous progress in recovering from the damage from the financial crisis," Yellen said at a panel discussion in New York on Thursday night. "Slowly but surely, the labor market is healing. For well over a year, we've averaged about 225,000 jobs a month, the unemployment rate now stands at 5%, and we're coming close to our assigned congressional goal of maximum employment."

The Fed's policy-tightening will be a gradual process given that risks to the U.S. economy remain, New York Fed President William Dudley said Friday morning. He said the U.S. economy will probably grow about 2% in 2016.

"Although the downside risks have diminished since earlier in the year, I still judge the balance of risks to my inflation and growth outlooks to be tilted slightly to the downside," Dudley said in prepared remarks.

The first-quarter earnings season will kick off in earnest on Monday afternoon when unofficial bell-ringer Alcoa (AA - Get Report) reports. Uncertainty over how major companies fared over the quarter kept bulls on the sidelines and pushed bears to sell.

"As we start to kick in to earnings season there's going to be anxiety," Matt Kaufler, portfolio manager at Federated Investors, told TheStreet. "There's a general skepticism that [earnings] are perhaps steady but they'll be unenthusiastic. Steady but uninspiring."

The early prognosis on companies' quarterly performances isn't promising. S&P 500 earnings are expected to fall 7.9% in the first quarter, their third straight decline and the worst  losing streak since mid-2009. Excluding the energy sector, earnings are forecast to drop 3.6%.