Over the years, the Postal Service has had to close offices, and has proposed stopping Saturday deliveries as a way to cut costs.

Though volume on Sundays is likelier to be slower than on other days, it shows Bezos' continued push to be more entrenched into consumers lives. Amazon has pushed itself as a leader in e-readers, with its popular Kindle devices, and the company has upped its game in the tablet market, with the recently revamped Kindle Fire HDX. Amazon is also reportedly working on a smartphone, with 3D imaging, as well as a set-top box, akin to Apple ( AAPL) TV.

Amazon has looked at several different ways to get consumers to buy more goods from it, outside of the aforementioned alternative delivery methods. It is expanding into delivering groceries, with its Amazon Fresh initiative, in the Seattle and Los Angeles area, and recently launched a Login & Pay service for websites, similar to what PayPal offers.

By offering deliveries on Sunday, Amazon is hoping to have more people sign up for its $79-a-year Amazon Prime service. The service allows for free two-day shipping, and also offers a wide variety of movies and television shows, akin to Netflix ( NFLX) services, which has more than 30 million users in the U.S.

Amazon does not publicly disclose Prime members, but it did note in last quarter's earnings release it signed up "millions of new Prime members," on the way to reporting $17.09 billion in revenue.

-- Written by Chris Ciaccia in New York

>Contact by Email.

If you liked this article you might like

Best Buy CEO -- We, Together With Amazon, Will 'Kill' the Competition

How to Invest Just Like a Millionaire

Kohl's to Accept Amazon Returns in 82 Stores and Yet Stock Does Nothing