I have found my mistake, the code below:

int x = j - (center+1), y = i - (center+1);

should be this:

int x = j - center, y = i - center;

The kernel is 5×5, then center = 2. Because of trying to shift the kernel window such that it centers around (0,0) — so it should range from -2 to +2. My mistake had it from -3 to +1, which is off by one and leads to asymmetric blur.