Globally it has been estimated that only one third of applied N is recovered in the harvested component of grain crops. This represents an incredible waste of resource and the overuse has detrimental environmental and economic consequences. There is substantial variation in nutrient use efficiency (NUE) from region to region, between crops and in different cropping systems. As a consequence, both local and crop specific solutions will be required for NUE improvement at local as well as at national and international levels. Strategies to improve NUE will involve improvements to germplasm and optimized agronomy adapted to climate and location. Essential to effective solutions will be an understanding of genetics (G), environment (E), and management (M) and their interactions (G x E x M). Implementing appropriate solutions will require agronomic management, attention to environmental factors and improved varieties, optimized for current and future climate scenarios. As NUE is a complex trait with many contributing processes, identifying the correct trait for improvement is not trivial. Key processes include nitrogen capture (uptake efficiency), utilization efficiency (closely related to yield), partitioning (harvest index: biochemical and organ-specific) and trade-offs between yield and quality aspects (grain nitrogen content), as well as interactions with capture and utilization of other nutrients. A long-term experiment, the Broadbalk experiment at Rothamsted, highlights many factors influencing yield and nitrogen utilization in wheat over the last 175 years, particularly management and yearly variation. A more recent series of trials conducted over the past 16 years has focused on separating the key physiological sub-traits of NUE, highlighting both genetic and seasonal variation. This perspective describes these two contrasting studies which indicate G x E x M interactions involved in nitrogen utilization and summarizes prospects for the future including the utilization of high throughput phenotyping technology.