Can LLMs Beat Classical Hyperparameter Optimization Algorithms?

This paper compares the performance of LLMs with classical hyperparameter optimization algorithms in tuning the hyperparameters of a small language model.