That problem is nearly always caused by the aboove ground check valve. But this check valve should be on the "inlet" side of the pressure tank for any of this to make sense. If so, then yes you need to remove the above ground check valve. However, this check valve is probably masking the real problem, which is that your down hole check valve is leaking back. The down hole check may not leak back after you remove the above ground check which puts full pressure on the bottom check, and may make it seal better. If after removing the above ground check valve the pressure leaks off when you are not using any water, the bottom check will have to be replaced. But you won't know until you remove the above ground check and see what the pressure does.