Even though current technologies allow for automated feedback, evaluating content and generating discourse-specific feedback is still a challenge for automated systems, which explains the gap in research investigating the effect of such feedback. This study explores the impact of automated formative feedback on the improvement of English as a second language (ESL) learners’ written causal explanations within two cause-and-effect essays and across pre- and post-tests. Pre- and post-test drafts, feedback reports for first and revised drafts from the automated writing evaluation system, and screen-capturing videos collected from 31 students enrolled in two sections of an advanced-low-level academic writing class were analyzed through descriptive statistics and the Wilcoxon signed-rank test. Findings revealed statistically significant changes in learners’ causal explanations within one cause-and-effect essay while no significant improvement was observed across pre- and post-tests. The findings of this study offer not only insights into how to further improve automated discourse-specific feedback but also pedagogical implications for better learning outcomes.