How do you improve data quality when working with Machine Learning?

There are several ways to approach Machine Learning, so I wanted to compare experiences.

I am particularly interested in data quality and performance. I would like to compare approaches that people have actually used rather than just theoretical suggestions.

What would you tell someone trying this for the first time?