Currently, chatbots based on generative AI are used by students often at their own initiative and unguided. Learners use chatbots primarily as static text generators, thereby underusing their potential. This experimental study investigates whether chatbot-initiated questions as a form of monitoring prompt can promote learning. It provides an analysis of chatbot interaction patterns, mental effort, and knowledge outcomes among 71 high school students. Two groups engaged in 15 minutes of research with a ChatGPT-4o-based chatbot, and the experimental chatbot ended each output with a comprehension or reflection question. Results show that chatbot-initiated questions increased reflective responses but did not significantly impact mental effort or knowledge gain. In terms of interaction, the control group rarely engaged in deeper conversation while in the experimental group 41% of all prompts were answers. This study demonstrates that lightweight system prompt interventions can help students go beyond using chatbots as static text generators.