We are interested in the potentials and pitfalls of introducing computer algebra systems in lower secondary school, investigating the case of the Faroese Islands. In order to identify what algebraic knowledge is tested in the final written exam in mathematics after the ninth grade, and how this would change if computer algebra systems were allowed at that exam, we analyse all exam exercises from the past 10 years in terms of the techniques required to solve the exercises both with and without symbolic tools. The comparison suggests that fundamental algebraic structures may not be learned if students consistently use computer algebra systems for the tasks given in the exam.