Temporal lobe epilepsy (TLE) and mild cognitive impairment (MCI) are both subject to intensive memory research. Memory problems are a core characteristic of both conditions and we wonder if there are analogies which would enrich the two distinct research communities. In this review we focus on memory decline in both conditions, that is, the most feared psychosocial effect. While it is clear that memory decline in MCI is highly likely and would lead to the more severe diagnosis of Alzheimer"s disease, it is a debate if TLE is a dementing disease or not. As such, like for MCI, one can differentiate progressive from stable TLE subtypes, mainly depending on the age of onset. Neuroimaging techniques such as volumetric analysis of the hippocampus, entorhinal, and perirhinal cortex show evidence of pathological changes in TLE and are predictive for memory decline in MCI. Several studies emphasize that it is necessary to extend the region of interest-even whole-brain characteristics can be predictive for conversion from MCI to Alzheimer"s disease. Electroencephalography is increasingly subject to computational neuroscience, revealing new approaches for analyzing frequency, spatial synchronization, and information content of the signals. These methods together with event-related designs that assess memory functions are highly promising for understanding the mechanisms of memory decline in both TLE and MCI populations. Finally, there is evidence that the potential of such markers for memory decline is far from being exhausted. Similar structural and neurophysiological characteristics are linked to memory decline in TLE and MCI. We raise the hope that interdisciplinary research and cross-talk between fields such as research on epilepsy and dementia, will shed further light on the dementing characteristics of the pathological basis of MCI and TLE and support the development of new memory enhancing treatment strategies.