The complexity of turbulent fluid motion often requires a model-reduction approach to dissect the interaction of different scales into a hierarchy of subprocesses. These subprocesses and the structures that are associated with them are often categorized by kinetic energy over a selected range of scales, but other criteria for an ordering are conceivable. The subsequent partitioning of retained and dismissed scales creates a general closure problem, where we have to model the influence of dismissed scales on retained scales.  Large-eddy simulations have long dealt with modeling the interactive effects of subgrid scales on the scales that can be resolved on a given grid. We will make a case that dynamic closure problems are inherently nonlocal in the retained structures. This argument will use Mori-Zwanzig (MZ) theory to formulate a self-contained dynamical system that does not suffer from scale truncation, but instead involves memory terms for the retained scales. Approximations of these memory terms will result in markedly more accurate closure techniques. We will carry over these findings to a set of nonlinear equations, arising from the harmonically balanced Navier-Stokes (HBNS) equations. Machine-learning techniques, inspired by the Mori- Zwanzig formalism, will be used to demonstrate the importance of flow-field history to arrive at highly truncated,  but properly closed model equations.  The implications on advanced turbulence models will be addressed as well.