We are interested in score reduction, specifically the task of adapting musical scores originally composed for another instrument (e.g. piano) into scores playable on the guitar. This process, traditionally performed by experienced guitarists, is complex and time-consuming. This study proposes to automate this task by framing score reduction as a combinatorial optimization problem under constraints and use Reinforcement Learning (RL) to solve it. The RL agent sequentially evaluates notes in the input score, deciding whether to keep or discard each note and selecting its position on the guitar fretboard. We use a graph representation for scores and utilize a transformer encoder to capture the state. The guitar's physical characteristics and the need to retain the musicality of the original score present challenging constraints. We design two reward functions to balance the trade-off between musicality and playability and train the agent using Proximal Policy Optimization (PPO).