Preheat your oven to 400°F (200°C). Line a baking sheet with parchment paper.
In a medium saucepan, bring the water and unsalted butter to a boil over medium heat.
Once boiling, add the all-purpose flour all at once. Stir vigorously with a wooden spoon until the mixture forms a ball and pulls away from the sides of the pan.
Remove the pan from the heat and let it cool for 5 minutes.
Add the eggs, one at a time, beating well after each addition until the dough is smooth and glossy.
Transfer the dough to a piping bag fitted with a large round tip.
Pipe the dough onto the prepared baking sheet into 3-4 inch lengths, leaving space between each éclair for expansion.
Bake in the preheated oven for 30-35 minutes, or until the éclairs are puffed up, golden brown, and crisp. Remove from the oven and let them cool completely on a wire rack.
While the éclairs are cooling, prepare the filling. In a saucepan, heat the whole milk over medium heat until steaming but not boiling.
In a separate bowl, whisk together the granulated sugar, cornstarch, and egg yolks until smooth and pale.
Slowly pour the hot milk into the egg mixture, whisking constantly to prevent curdling.