I would think that most of us have trouble keeping our mind focused while in prayer. It can become frustrating and even disheartening to want to give your whole heart, mind, and soul to God one moment, and then be struggling with a sinful imagination the next. Here is a very helpful bit of wisdom from Brother Lawrence on how to combat that struggle.
“One way to re-collect the mind easily in the time of prayer, and to preserve it more in tranquility, is not to let it wander too far at other times. You should keep your thoughts strictly in the presence of God. And being accustomed to think of Him often, you will find it easy to keep your mind calm in the time of prayer, or at least to recall it from its wanderings.”