What is step size?

Step size refers to the increment or interval used in a numerical method or algorithm to move from one point to another in a given space, often in iterative processes such as optimization or numerical integration. In machine learning, for instance, it can denote the learning rate, which determines how much to adjust the model's parameters in response to the estimated error. A smaller step size may lead to more precise results but can also require more iterations, while a larger step size can speed up convergence but may risk overshooting optimal points.