The sensor in the E-M1 is sometimes mildly complained about by Olympus and other users for noise, even at the default ISO200 setting, and more so at higher settings. Auto ISO also. Most often appearing in shots with an expansive clear sky.
Somewhere on a forum, I was picked up on this about one of my displayed images, and while mild noise has never bothered me too much, I listened to his suggestion - "whenever you can, use ISO Low setting", which I believe is actually ISO100.
Well, I have been following his advice since, and I have to say the images are much cleaner and smoother, not just for clear blue skies. Now I rarely use ISO Auto, and have Noise Control (or whatever it is called), OFF. It also helps if you have to crop heavily.
Worth a try.
Cheers,
Don