We use it when something is done in an orderly manner. For example, you can learn how to read step by step: first, you learn the alphabet. Second, you learn how to read separate letters. Finally, you read words and then whole sentences.
Definition: carefully and orderly, done in a specific sequence.
Example: this task might be difficult. I'll tackle the task step by step.