This setting only applies to UWP app, which are default Windows apps or installed from the Store.
UWP was designed to suspend apps, when they're not running in the foreground, in order to save battery life. If your app is suspended, then it's paused at the CPU level and not checking for new data (like a weather or news app).