There are a number of instances when you cannot remove a replica from Open Directory using Server Admin. This can be due to a variety of reasons, but if you cannot remove the replica from Server Admin then try doing so using slapconfig. For example, if the replica has an IP of 10.10.10.10 then the command would be:
slapconfig -removereplica 10.10.10.10