Stocks remained lower by mid-afternoon Tuesday as crude oil slid for the fourth trading day in a row. 

The S&P 500 was down 0.3%, the Dow Jones Industrial Average fell 0.41%, and the Nasdaq dropped 0.2%.

Crude oil continued to decline, its fourth straight day of losses, despite increased demand expectations from the International Energy Agency. The energy watchdog revised its demand forecast to 1.3 million barrels a day over this year from 1.2 million barrels, an increase led by manufacturing growth in India and China. Crude has fallen to multi-year lows in the past year as supply ballooned and demand weakened.

West Texas Intermediate crude oil was down 0.8% to $48.49 a barrel on Tuesday. Crude topped its best level in nearly a year last week and has been in retreat ever since.

The two-day Federal Open Market Committee meeting, which kicked off Tuesday morning, will conclude with a statement and fresh forecasts on Wednesday afternoon followed by a press conference helmed by Federal Reserve Chair Janet Yellen. Chances of a June rate hike are virtually non-existent, but investors will be on high alert for any clues as to when another increase could come.

"The burden of proof is now on the June jobs report to right the ship and rekindle even the faintest anxiety over the Fed's intentions," said David Joy, Ameriprise chief market strategist. "It should be noted that Fed Chair Yellen did strike a somewhat optimistic tone last week regarding the economy. The latest round of reports has suggested a decent rebound from the first quarter, including a rise in manufacturing activity, housing and consumer spending."

According to CME Group Fed funds futures, a July rate hike currently has a 23% probability, September a 37% probability, November a 38% probability, and December a 59% probability.

European markets were also lower for their second session this week as the likelihood of a British exit from the European Union became more likely. Great Britain will vote in a referendum in nine days and the "leave" option currently has a seven-point lead among voters in the latest poll. A "Brexit" would have major economic implications for the stability of the EU.

Retail sales in May climbed 0.5%, a touch above estimates of a 0.3% increase though a significant slowdown from 1.3% growth in April. Stripping out the effects of autos and gas, core retail sales increased 0.3%.

"The composition of spending in May was ... encouraging," RBS Securities economists wrote in a note. "In particular, spending at restaurants (a key discretionary spending category) advanced by 0.8% on top of a 0.5% rise in April. Spending on apparel was up 0.8%, building on a 0.4% increase in April."

U.S. import prices rose 1.4% in May, according to the Bureau of Labor Statistics. The reading came in above an expected 0.8% increase. Export prices rose 1.1% in May.

Whole Foods Market (WFM) slipped more than 2% after the Food and Drug Administration sent a warning regarding violation at its Everett, Mass., manufacturing plant. Regulators noted a number of "serious" violations on regulations for packing and handling food. 

Gunmakers pulled back after Monday's rally with Smith & Wesson (SWHC) and Sturm Ruger (RGR - Get Report) sharply lower. The gunmakers had spiked on Monday, a day after the shooting in Orlando, Fla., on expectations talk of gun control might lead to a bump in sales. 

Alibaba (BABA - Get Report) climbed after co-founder Jack Ma said the company would no longer rely upon the closely watched gross merchandise volume sales metric after U.S. regulators began an investigation into the practice. The Chinese e-commerce company also said it expects 48% revenue growth for its fiscal year ending in March.

NXP Semiconductors (NXPI - Get Report)  fell after reaching a deal to sell its Standard Products business to a consortium of financial investors consisting of Beijing Jianguang Asset Management and Wise Road Capital for about $2.75 billion. The division generated annual revenue of $1.2 billion over fiscal 2015. The transaction is expected to close in the first quarter of 2017.

NXP Semiconductors is a holding in Jim Cramer's Action Alerts PLUS Charitable Trust Portfolio. Want to be alerted before Cramer buys or sells NXPI? Learn more now.

Baidu (BIDU - Get Report) slumped after cutting its sales forecast for the second quarter on a drop in medical advertising revenue. Chinese Internet companies have recently been under increased scrutiny over health care advertising after a college student allegedly died from medical misinformation published on Baidu. The case is under investigation by Chinese officials.

The Chinese Internet giant expects revenue in the period of 18.1 billion yuan to 18.2 billion yuan ($2.81 billion to $2.82 billion), down from earlier projections of 20.11 billion yuan to 20.58 billion yuan.