As per the Food and Agriculture Organization (FAO), agricultural productivity needs to be increased by 70% to feed a projected 10 billion people by the year 2050, and fertilizer application plays a key role in achieving this goal. Throughout the world, fertilizer usage has significantly increased to improve crop yields. Unfortunately, several studies have shown that over 65% of the fertilizer is being wasted leading to various environmental problems such as nitrogen runoff into lakes, rivers, and oceans. In addition, the traditional practice of uniform fertilizer application without regard to field and/or crop conditions can result in wasted fertilizer. To address these challenges, we propose a simple Q-learning-based simulation tool for studying the dynamic application of fertilizer. Q-learning is particularly well suited for solving this problem, as the framework allows sensing the environment to characterize the site and crop conditions and determine the amount of fertilizer to apply. We used remote sensing data as a proxy for crop health monitoring. We used reward shaping to determine the optimal amount of fertilizer. We compared our framework with other popular deep-learning approaches Deep Q Network (DQN), Double Deep Q Network (DDQN), and Dueling Network (Due_N). Experimental results show that our approach is computationally efficient while matching or performing better than other approaches.