Let me take a guess - your lever travels noticeably more before it engages the brakes.
Normally, when you release the brake lever, the rubber seals pull the pistons back just a little. This is not very noticeable because the pistons are settled in the position close to the pads. So, next time you squeeze the lever, it has to travel a bit before you start feeling the pressure.
You probably pushed pistons in when you were changing pads/removed calipers. This means that the pistons are a little bit further away from the pads than before. As you release the lever, the seals pull them back. And seemingly there is no way to to get them closer to the pistons.
If these are your symptoms, let me know - there is an easy fix, and there is nothing wrong with your braking system.
A different take: if your bike is at least a few years old, and you never rebuilt the calipers, I am pretty sure some pistons are binding more than others. This means when you release the brake lever, on pad is pulled back by the seals as explained above, while another pad is not - this one drags. Not really a big issue.
In any event, you can check how bad the drag actually is. Ride for a few miles without using the front brakes. Stop and check the temperature of your rotors. If one is very warm or even hot - you got a somewhat critical problem. Otherwise, not a big issue.