Music classification is a prominent research area within Music Information Retrieval. While Deep Learning methods are capable of adequately performing this task, their classification space remains fixed once trained, which conflicts with the dynamic nature of the ever-evolving music landscape. This work explores, for the first time, the application of Continual Learning (CL) in the context of music classification. Specifically, we thoroughly evaluate five state-of-the-art CL approaches across four different music classification tasks. Additionally, we showcase that a foundation model might be the key to CL in music classification. For that, we study a new approach called Pre-trained Class Centers, which leverages pre-trained features to create dynamic class-center spaces. Our results reveal that existing CL methods struggle when applied to music classification tasks, whereas this simple method consistently outperforms them. This highlights the interest in CL methods tailored specifically for music classification.