If you don’t want to use milk, or don’t have the money to go get milk, you can use butter to make your frosting instead. One recipe calls for six tablespoons of whipped butter (you can use the whisk attachment on a power mixer to do this).
Then you add powdered sugar and water, instead of the milk. This is one of the easiest ways to eliminate milk from frosting. For a milk-based frosting recipe, this will not work. You should avoid these kinds of frosting recipes if you don’t want to use milk, or think it’ll taste weird with a milk substitute like soy or almond milk. If you’re lactose intolerant, you could simply replace the milk with lactose free milk (which is available in many grocery stores).