Cisco states that PPPoE is only supported on the 5500x ASA models in single routed mode without failover. (see here). When a telco tech came and replaced a DSL modem with a transparent bridge several hundred miles from my office, they gave me little choice but to challenge Cisco on that statement. I configured the VPDN group and tied it in with the outside interface and everything was working like it was before. Or was it?
First lesson learned – the ASA will allow you to configure PPPoE on an interface when in a failover pair. There are no warning lights, no flashing alarms, it will just let you do it. And it might even work, like it did for me.
Or you might end up in a strange situation where the standby firewall has gone through the PPPoE stages and now has a session up. In my case, this was identified with the following command:
‘show vpdn session state pppoe’ (or something like that)
My active firewall was at the PADI_SENT stage and was stuck there for a long time. My standby firewall was at the SESSION_UP stage and had an active Internet connection over the DSL line. This is perplexing to say the least. I fixed it by shutting the switchport that connected to the WAN interface on my standby firewall. Then I stripped the VPDN configuration from the outside interface on the active firewall and re-applied it. After confirming that the state was SESSION_UP on the active firewall, I no shut the port on the standby firewall. The end result was that the DSL connection was up on the active firewall and the standby firewall was failover ready. I thought I had out-smarted Cisco, I should have known better.
Lesson two – an ASA will disable failover if it boots up and has PPPoE attached to an interface.
Of course these lessons always happen at inconvenient times and/or places. I learned this when I was upgrading the ASA code. I had done it dozens of times and expected this time to be no different. I copied the new code over to the firewall pair, verified the sha-512 signature, changed the boot variables, wrote memory, and reloaded the standby unit. I watched the clock tick for several minutes. After 10 minutes and the active unit still reporting that the standby unit had failed, I knew something had gone wrong.
I got in touch with someone on-site who confirmed that both firewalls had green active lights. Fantastic. I had them immediately power down the secondary unit until I could get eyes on the console output during a boot sequence.
The firewall boot looked normal – strange. I logged in and found out that failover was OFF! This can cause some serious issues that are very hard to detect and troubleshoot (especially from hundreds of miles away). One example is that any device which uses the firewall as a default gateway now has two devices responding to their ARP request. This is a race condition and any device that caches an ARP reply from the secondary/should-be-standby-but-is-active unit is in for a bad time. Good luck getting outside of your subnet, I hope they don’t cache that MAC for too long.
When I tried to enable failover, I got the message:
PPPOE Client cannot be enabled on interface, Gi1/1(outside)
failover is not compatible with above configurations,
user must manually remove or fix them as instructed before failover can be enabled.
The fix for this was to remove the ‘ip address’ and ‘pppoe client’ commands from the interface mentioned in the error message, then try again. I ended up removing those commands from both firewalls before finally enabling failover on the secondary unit, which did sync and settle into standby.
TL;DR – don’t configure PPPoE on an active/standby pair of ASAs.