Stuttering (mouse, keyboard input, video) in 5.3/5.4 when swap is being used.

[Andrei Borzenkov <arvidjaar@gmail.com>]

Recently Ubuntu released kernel 5.3 for 18.04 LTS and I immediately
started observing periodic stalls/freezes without any change in my
normal usage pattern. The effects I noticed:

- mouse is frozen for couple of seconds
- keyboard input is frozen for couple of seconds
- video freezes for a couple of seconds

After some searching and experiments all those freezes look associated
with swap usage. To make it absolutely clear - when this happens there
is *NO* memory shortage. The situation is really - as soon as _some_
swap is being used, I start to see observed effects. It is enough to
have several megabytes of used swap for this. I have a lot of free
memory when this happens. Like

              total        used        free      shared  buff/cache
available
Mem:           7.7G        1.7G        3.6G        419M        2.4G
  5.2G
Swap:           15G        218M         15G


This is trivially demonstrated by removing swap when stuttering is
observed. Without any change in workload stuttering immediately
disappears. If I add swap back, I again observe stuttering as soon as
even minimal amount of swap is allocated.

I tested also 5.4.13 which shows the same problem. From practical point
of view it seems to be less eager to use swap, so at first it looks like
problem is fixed; but if I force it to allocate swap (by starting
several VM as example) I again see the same freezes.

I then went back to 5.0 and problem is gone. No matter how many
processes I start there are no freezes. I do see obvious slowdown due to
increased disk activity as memory consumption grows, but this is expected.

To me this looks like rather serious regression. There seem to be quite
a lot of posts about similar issues starting with 5.3.

I understand that all this is pretty much unscientific. I am happy to
collect whatever information may be required (including compiling custom
kernel).