It is believed that Alcuin of York (c. 735 - 804 AD) is the author of Propositiones ad Acuendos Juvenes, a collection of 53 recreational problems, some of them well known.
Another topic to look into is calendar mathematics, which was important in this time period. According to Victor Katz, Charlemagne recommended that Church Schools include the mathematics necessary for Easter computation.