Yes, it is important to market offline as well as online via e-commerce. You can use offline marketing to drive people to your e-commerce site. Every piece of paper your business uses should be thought of as a marketing tool. For example, business cards, company stationery, invoices, receipts, fax covers and more should include your website address.
People also love free (to them) promotional items such as pens or key chains. Just make sure your e-commerce site is included on the promotional item. You can also drive business to your site by sponsoring events such as local little league football, baseball, or soccer teams.