Banging my head against an IPSec tunnel issue

VPN
1

I’ve been going in circles trying to troubleshoot an IPSec tunnel and realized I need another set of eyes and some suggestions on where to look.

​

Background: Set up a site to site tunnel in early August, ran a test vpn ike-sa gateway XXX and test vpn ipsec-sa tunnel XXX and everything came up fine. Walked away because we didn’t need it at the time.

I come back to this now (when we need it) and I can’t get the tunnel to come up again. Nothing changed on the firewall configs.

SO…

I went ahead and rebuilt the tunnel from scratch, same config settings, nada. I’ve spent a few days on this.

I can see an ipsec-esp-udp flow between both firewalls in the session browser, but ike and ipsec never establish.

Crypto profiles are the exact same between devices.

I have pasted config from both firewalls below. On Firewall A there are a bunch of other tunnels all working perfectly fine…

Any thoughts on where to look next? Not that it matters but this is between a PA-820 and a PA-850.

Firewall A:
ike gateway

FirewallB_DC {
authentication {
pre-shared-key {
key -
}
}
protocol {
ikev1 {
dpd {
enable yes;
}
}
ikev2 {
dpd {
enable yes;
}
ike-crypto-profile IKEv2-AES256-SHA;
}
version ikev2-preferred;
}
protocol-common {
nat-traversal {
enable yes;
}
fragmentation {
enable no;
}
passive-mode yes;
}
local-address {
interface ethernet1/3;
ip ;
}
peer-address {
dynamic;
}
peer-id {
id [email protected];
type ufqdn;
}
local-id {
id ;
type ipaddr;
}
}

IPSEC:
FirewallB_S2S {
        auto-key {
          ike-gateway {
            FirewallB_DC;
          }
          ipsec-crypto-profile default;
        }
        tunnel-monitor {
          enable no;
        }
        tunnel-interface tunnel.30;
        enable-gre-encapsulation yes;
      }

​

Firewall B
gateway {
      PIT_DC {
        authentication {
          pre-shared-key {
            key - <key>
          }
        }
        protocol {
          ikev1 {
            dpd {
              enable yes;
            }
          }
          ikev2 {
            dpd {
              enable yes;
            }
            ike-crypto-profile IKEv2-AES256-SHA;
          }
          version ikev2-preferred;
        }
        protocol-common {
          nat-traversal {
            enable yes;
          }
          passive-mode no;
          fragmentation {
            enable no;
          }
        }
        local-address {
          interface ethernet1/1;
        }
        peer-address {
          ip <public ip>;
        }
        local-id {
          id [email protected];
          type ufqdn;
        }
        disabled no;
        peer-id {
          id <public ip>;
          type ipaddr;
        }
      }
    }
  }

IPSEC:
ipsec {
      FirewallA_S2S {
        auto-key {
          ike-gateway {
            FirewallA;
          }
          ipsec-crypto-profile default;
        }
        tunnel-monitor {
          enable no;
        }
        tunnel-interface tunnel.30;
        enable-gre-encapsulation yes;
      }
    }

Firewall B has an address assigned via DHCP.

Firewall A ikemgr.log shows nothing from Firewall B, and Firewall B’s log shows it sending the init and then just sitting there forever before timing out.

Given that this is fairly simple I feel like I must be overlooking something really stupid.

Any thoughts?

Thanks!

2

I’ve had AT&T block esp traffic twice before for one of my sites, and this looks very similar. I’d give your ISP a ring for firewall b

3

Do you have a rule permitting the traffic to build the tunnel?

4

Debug it -

‘debug ike gateway gatewayname on dump’

‘tail follow yes mp.log ikemgr.log’

Remember to turn off the debug -

‘debug ike gateway gatewayname off’

Replace ‘gatewayname’ with the name of the gateway.

5

Is NAT traversal required? You have it on.

6

Have you rebooted either end, if that’s possible, understand sometimes its not.

Have you cleared the sessions on both ends?
clear vpn ike-sa gateway ‘gatewayname’
clear vpn ipsec-sa tunnel ‘tunnelname’

I don’t use this on my tunnels. enable-gre-encapsulation yes;
Make sure the default ipsec profiles on both ends match, default can be changed.
I prefer to force IKEv1 or v2, usually V2 if PA to PA.

}

local-id {

id [email protected];

type ufqdn;

}

disabled no; >>>>> MY WORKING CONFIG DOESNT HAVE THIS disabled no

peer-id {

id ;

type ipaddr;

7

I would try to set the peer as an Ip and retest just to see if it’s resolution issue for the fqdn.

If still the same I would perform a pcap both sides can you see the traffic sent and received on the peer?

8

I dont know if you issue is solved but why you are using GRE encapsulation. Your config doesnt need it. Dont tick unnecessary things.

And share system logs as well, whats the error tells about the issue.

9

Did you find and fix your problem OP?

10

I’ve run into exactly this. ATT firewall was doing NAT-T, none ATT was sending raw ESP. Once we figured out ESP was being filtered on the ATT side we set the none ATT to NAT-T (even though everything had routable IPs) and everything started working.

There was much fist waving at ATT that day.

11

Maybe?

The interface is in the untrust zone, so any/all traffic is allowed. I do have rules once the tunnel is established.

Where I’m scratching my head is we don’t have any explicit rules allowing this for any of our other tunnels either.

On a whim I created a rule from untrust to untrust allowing ike/ipsec etc. It didn’t make a difference.

Thoughts on what I’d need to allow?

12

On a PA 440 running 10.2.9-h1 the syntax of the tail command should be:

tail follow yes mp-log ikemgr.log

13

Yes, the one branch location is natted.

14

No, but was able to get the firewall swapped and into a secondary circuit in our DC so we can test setting it up on two known good circuits. If that works then it’s the ISP.

15

UDP 500/4500 or IKE and IPSec application. If you turn logging on in the rule, make sure you can see the traffic in your traffic logs. I’d also expect to see an IKE negotiation started log in your system logs. It’s those IKE logs that will point you in the right direction once you’ve confirmed you actually have two way connectivity.

If I was troubleshooting this, first I’d make sure I could ping between the firewalls. Then I’d make sure I could see the outbound IKE traffic. Then I’d check for any IKE logs.

I’ve also just had a thought. Are the firewalls on different PAN OS versions? Particularly one before 9.1.7 and one on 9.1.7 or later by any chance?

16

I have seen the nat work of 1 site starts , but it will not work in the other direction . Make sure the nat traversal check box is checked on both sides.
Also clear the sessions in the session browser on both sets of firewalls .
Use the clear vpn and test vpn on both sides after that .

17

May i know if the issue was fixed ?

18

Yes, different versions. I think one is 10.0.4 and the other is 10.1.0.

I should try to update the 10.0.4 to 10.1.2, get them on the same version and go from there.

Both can ping. It’s a strange issue!

19

Yes we got it fixed. We changed the crypto type and rebuilt. It came right up.

20

Never say you have problem when your running 10.1.0 . Upgrade that to 10.1.2 lol