2
$\begingroup$

When fitting neural networks, I often run stochastic gradient descent multiple times and take the run with the lowest training loss. I'm trying to look up research literature on this practice, but I'm unaware what its called. Any terms, keywords, or references are appreciated.

The closest thing I have found is "Stochastic Gradient Descent with Restarts", but I don't believe its quite the same idea.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
2
$\begingroup$

A common name for running a local optimization algorithm multiple times in order to get closer to a global minimum is "Multi Start Methods". This is studied in the field of Global Optimization. For a quick reference, see the MATLAB documentation on its global optimization toolbox.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.