When a layer is not required you can delete it. When deleting a layer, it and all of the objects contained in the layer are removed from the canvas. Layers are deleted by clicking on the button. Of course this will only affect the currently selected layer.
Doing so on regular basis is the better way to keep your diagram clean and light for when you want to export it to another format other than the native Dia format.