Combine the dry ingredients. In a medium bowl, whisk together the all-purpose flour, baking powder, and salt until evenly mixed. Set aside.
Cream the butter and sugar. In a large mixing bowl, beat the softened butter and granulated sugar with an electric mixer for 2–3 minutes, until the mixture looks light and creamy.
Add the egg and flavorings. Beat in the egg, vanilla extract, and almond extract, if using. Mix until smooth, scraping down the sides of the bowl as needed.
Make the cookie dough. Add the dry ingredients gradually, mixing on low speed until a soft dough forms. Stop mixing as soon as the flour is incorporated to keep the cookies tender.
Chill the dough. Divide the dough into two portions and flatten each into a disc. Wrap tightly and refrigerate for at least 1 hour. Chilling makes the dough easier to roll and helps the cookies maintain their heart shapes while baking.
Prepare the oven and baking sheets. Preheat the oven to 350°F (175°C). Line two baking sheets with parchment paper.
Roll and cut the cookies. Lightly flour a clean work surface and rolling pin. Roll one portion of dough to approximately ¼-inch thickness. Cut out hearts using cookie cutters and transfer them to the prepared baking sheets, leaving about 1 inch between cookies. Gather and reroll the scraps as needed.
Bake the cookies. Bake for 8–11 minutes, depending on the size of the cutters, until the edges are set and just beginning to turn lightly golden. The centers should remain relatively pale.
Cool completely. Let the cookies rest on the baking sheets for 5 minutes before transferring them to a wire rack. Allow them to cool completely before decorating.
Prepare the icing. In a small bowl, whisk together the powdered sugar, 2 tablespoons of milk, and vanilla extract until smooth. Add more milk a little at a time until the icing reaches a spreadable consistency. Divide the icing into separate bowls and tint with gel food coloring if desired.
Decorate and serve. Spread or pipe the icing over the cooled heart-shaped cookies. Add sprinkles while the icing is still wet. Allow the icing to set before stacking or storing the cookies.
