i'd think of some broken routing and/or rpf. For example if you're not announcing the route to your provider on the inactive router, the provider might be filtering traffic coming from you because of RPF.
diag:
run traceroute to the IP to see where the traffic to the (inactive router) IP goes and where it stops answerring,
check on the last hop that is answerring that the route TO your inactive router is correct,
then check the reverse path: where is the route to your OUTSIDE ip (from where you are pinging) pointing to? what kind of devices are there, who owns them? can you traceroute from the inactive device to the outside? Whats the path?
come back with these details if that steps don't enlighten you by yourself.