We can't totally say New Zealand is a cheap place to live. But if you compare the standard of living in New Zealand with other European or North America countries, you will definitely see the difference. There are definitely some places, cities in New Zealand with affordable cost of living. If you are a serious type and hardworking you would be able to afford the cost of living there because the country is filled with a lot opportunities.
You can find places like Rotorua, Napier/Hasting, Palmerston North, Hamilton, Tauranga very affordable to live in. In Rotorua you can find cheap rent, food, entertainment. There are couples of industries in the city. If you like nature very well, you will find the experience really awesome. Other places are also cheap to live in with great opportunity. They are good site for tourism, high mountain and landscape formation that attract tourists are there