The Insecure Wire

a network engineers perspective

Setting up AD Auth with Hashicorp Vault

Hashicorp Vault is open source and can be used in DevOps processes for secure automated retrieval of keys and secrets.
I recently setup Vault as a password / key store. Here is how to configure Vault for Active Directory LDAP authentication.

This setup assumes the following:
‘sAMAccountID’ is the username attribute within AD for the user/s you want to authenticated to Vault.
The user must be a member of a specific group to be granted access to the Vault secrets path.
Vault is installed and initialized with the root token.

1. Create a file named IT.hcl with the following Vault policy as its contents:

path "secret/data/IT" {
capabilities = ["create", "read", "update", "delete", "list"]
}

2. Write the policy into the Vault:

vault policy write IT IT.hcl

3. Enable LDAP Auth:
vault auth enable ldap
4. Write the LDAP auth config (edit the values for your binddn, groups and server name):

vault write auth/ldap/config \
url="ldap://server.domain.name" \
userattr="samaccountid" \
userdn="ou=Users,dc=domain,dc=name" \
groupdn="ou=Groups,dc=domain,dc=name" \
groupfilter="(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={{.UserDN}}))" \
groupattr="samaccountid"
binddn="cn=vault,ou=users,dc=domain,dc=name" \
bindpass='My$ecrt3tP4ss' \
upndomain="domain.name"

5. Map the Vault IT policy to the IT AD group:

vault write auth/ldap/groups/IT policies=IT

Note that in AD the group should be named ‘IT’ (for this example)
6. Test Vault AD Authentication:

vault login -method=ldap username='myUser'

7. Confirm your AD user has the permissions set in the IT Vault policy:

vault token capabilities secret/data/IT

In this example the AD user myUser is a member of the AD group ‘IT’ which has full permission to the /secret/data/IT Vault. All done 🙂

Palo Alto Networks Powershell Backup Script

As we recently rolled out a bunch of PA firewalls, I have created a powershell script to backup the running configuration using the XML API.

You can grab the script from my github here:
panbackup

The instructions are as follows:
1. Create the folder c:\panbackup\ on your Windows Server.

2. Create a local administrator on the firewall as a member of super users (read only). This will allow rights to export the full configuration with phash keys. Which means you can restore the config on a new appliance easily. The below PA documentation details how to create a local firewall administrator: Create Firewall Administrator

3. Generate your API key as follows: https:///api/?type=keygen&user=&password= You can also generate api key via cURL as per the PA documentation below:
Generate your API key

4. Test your powershell script! You may need to set the correct saving path, file names etc. Add a scheduled task and viola! Peace of mind.

ELK Stack with Palo Alto Firewall – Using Curator to clear indexes

I recently deployed an ELK stack (Elasticsearch, Logstash, Kibana) VM as logger for a Palo Alto Networks firewall. ELK is open source and allows you to create beautiful dashboards in Kibana.
I followed the following guide for integrating PAN firewall with ELK palo-alto-elasticstack-viz.

Overview Dashboard
Threat Dashboard

The issue I was having is that Elastic indexes would continue to grow and the VM would eventually run out of disk. To solve this problem I did the following:

1. Change to daily indexes, base on date stamp. Edit logstash config like so (this edit follows on from the above PAN-OS.conf logstash configuration file):

output {
if "PAN-OS_Traffic" in [tags] {
elasticsearch {
index => "panos-traffic-%{+YYYY.MM.dd}"
hosts => ["localhost:9200"]
user => "elastic"
password => "yourpassword"
}
}
else if "PAN-OS_Threat" in [tags] {
elasticsearch {
index => "panos-threat-%{+YYYY.MM.dd}"
hosts => ["localhost:9200"]
user => "elastic"
password => "yourpassword"
}
}
else if "PAN-OS_Config" in [tags] {
elasticsearch {
index => "panos-config-%{+YYYY.MM.dd}"
hosts => ["localhost:9200"]
user => "elastic"
password => "yourpassword"
}
}
else if "PAN-OS_System" in [tags] {
elasticsearch {
index => "panos-system-%{+YYYY.MM.dd}"
hosts => ["localhost:9200"]
user => "elastic"
password => "yourpassword"
}
}
}

Logstash will now create an index based on date stamp for the firewall log inputs.
2. Use Elastic Curator cli tool to create a shell script and run it with crontab:
Create /etc/curator/config.yml

client:
hosts:
- 127.0.0.1
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth: elastic:yourpassword
timeout: 30
master_only: False

logging:
loglevel: INFO
logfile:
logformat: default
blacklist: ['elasticsearch', 'urllib3']

Create /etc/curator/delete-after.yml
Set unit_count to the number of days to keep indexes. In my example anything older than 60 days gets deleted.

actions:
1:
action: delete_indices
description: >-
Delete indices older than X days (based on index name), for panos-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: panos-
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 60

Create /etc/curator/cleanup.sh and paste in:

#!/bin/bash
# This df command grabs the free space of the root '/'.
disk=$(df -H | grep -vE '^Mounted| /.' | awk '{ print $1 " " $5 " " $6 }' | awk 'NR == 2' | awk '{print $2}' |sed 's/%//')

# Delete indices older than 60 days.
curator --config /etc/curator/config.yml /etc/curator/delete-after.yml
echo $disk

Now add to crontab – to run the script 5 mins past midnight every night:

sudo crontab -e
5 0 * * * /etc/curator/cleanup.sh

That it! You can tweak the unit_count days if you want to have say only 7 days worth of logs depending on your use case. You can also run curator manually like so:

sudo curator --config /etc/curator/config.yml /etc/curator/delete-after.yml

This helps when debugging your script logic and checking that elastic is actually deleting indices.

Request a SAN certificate using MS CA Web enrollment Pages

1. Run these commands on the MS CA server:

certutil -setreg policy\EditFlags +EDITF_ATTRIBUTESUBJECTALTNAME2
net stop certsvc
net start certsvc

2. In the Attributes box, type the desired SAN attributes. SAN attributes take the following form:

san:dns=dns.name[&dns=dns.name]

For example : To add two DNS names to the SAN field , you can type:

san:dns=corpdc1.fabrikam.com&dns=ldap.fabrikam.com

Renaming a vSwitch in VMWare ESXi

At the ESX Console, log in and hit Alt-F1 then type unsupported and hit Enter. You won’t see the word “unsupported” appear as you type it but upon hitting Enter, you’ll be prompted for the root password. Type it in and hit Enter.

You be presented with the ESXi command prompt. The default text editor in ESXi CLI is vi:

cd /etc/vmware
vi esx.conf

Search for “name” using Esc, /name, Enter and keep hitting n (next) until you find the incorrectly named vSwitch. Change the word by hitting Esc, cw followed by the correct name, followed by Esc.

/net/vswitch/child[0001]/name = “SAN-A“

If you’re happy the name has been changed correctly in esx.conf, hit Esc, :wq! and hit Enter to write the changes back to disk and quit vi.
Back at the Linux prompt, type clear to clear the screen, and type exit and hit Enter to log out of the console. Alt-F2 will close the “Unsupported Console” returning you back to the black and yellow ESX Console.

Esc to log out, then finally F11 to restart the host.

Note I have tested this on ESXi 6.7u1 – you must restart after the change to esx.conf – other changes via UI do not work until ESXi is reloaded.

Configure iSCSI Port Binding using ESXCLI

1. Enable SSH console from the management settings > troubleshooting options on the ESXi console.
2. Bring up the CLI console with alt + f1.
3. Enter the following commands to bind the VMKernel port(s) to the software iSCSI adapter (ESXi 6.7).


~ # esxcli iscsi networkportal add --nic vmk1 --adapter vmhba64
~ # esxcli iscsi networkportal add --nic vmk2 --adapter vmhba64

Deploying Dell vLT on OS10

I was recently asked to assist our training environment with the deployment of mid market Dell data center utilizing OS 10 and Virtual Link Trunking. VLT is Dell’s implementation of multi-chassis etherchannel, similar to Cisco’s Virtual Port Channel. OS10 is an open networking platform, runs on Debian Linux and boots of the ONIE (Open Network Install Environment) boot loader.

For this environment / lab here is our bill of materials:
2 x Dell S4112F-ON
2 x Dell R640 1 RU servers
1 x Dell sc3000v2 SAN
1 x Palo Alto Networks PA-220

Dell Data Centre Design

Dell Data Centre Design

Dell vLT allows multiple switches to appear as one logical unit. VLT has layer 3 routing features and as such out of box can replace FHRR protocols such as VRRP. In this design there is a pair of S4112F-ON switches connected at 100gb/s. The left side is PVST+ root bridge for all VLANs and it is also the primary vLT peer. One thing to note in regard to the Dell vLT peer-routing feature. Both L3 SVI IP addresses will perform IP routing for a subnet, if the primary switch in a VLT peer goes down then the secondary will transparently respond to IP routing request. You can not ping the primary gateway IP when this fail over scenario is in affect. Note that vLT port-channels require a mirror of configuration on both switch members. This is regardless of access or trunk mode port-channels. There is an example below.

Here is the OS10 (10.4.2.2.265) configuration to achieve the above vLT design:
Core-1

vlt-domain 1
backup destination 192.168.x.2
discovery-interface ethernet 1/1/14-1/1/15
peer-routing
primary-priority 1
vlt-mac 00:11:22:33:44:56

interface port-channel3
description PA-Internet-Uplink
no shutdown
switchport access vlan 550
vlt-port-channel 3

interface ethernet1/1/7
description Link-1-to-PA
no shutdown
channel-group 3

Core-2

vlt-domain 1
backup destination 192.168.x.1
discovery-interface ethernet 1/1/14-1/1/15
peer-routing
vlt-mac 00:11:22:33:44:56

interface port-channel3
description PA-Internet-Uplink
no shutdown
switchport access vlan 550
vlt-port-channel 3

interface ethernet1/1/7
description Link-1-to-PA
no shutdown
channel-group 3

VMware ESXi Setting iphash load balancing via CLI

I recently setup x2 new ESXi 6.7u1 servers without access to each hosts LAN. I needed to configure the correct vSwitch failover setting so that the port channel uplinks would work correctly:

1. Enable SSH console from the management settings > troubleshooting options on the ESXi console.
2. Bring up the CLI console with alt + f1.
3. Enter the following commands to set the vSwitch and Management portgroup failover policy:

esxcli network vswitch standard policy failover set -v vSwitch0 -l iphash
esxcli network vswitch standard portgroup policy failover set -l iphash -p 'Management Network'

4. alt + f2 will switch you back to the vMware DCUI console.

Cisco ASA Dynamically open MS-RPC Ports

OK so this one is simple once you know how. TCP Port 135 (MS Remote Procedure Call Endpoint Mapper) requests high range ports > 1024 for Windows client / server networking. To allow this traffic across the ASA you need to pinhole the ports with the global policy map:

policy-map type inspect dcerpc dcerpc_map
parameters
timeout pinhole 0:10:00
class-map dcerpc
match port tcp eq 135

policy-map global_policy
class dcerpc
inspect dcerpc dcerpc_map

As well as permitting the traffic through the firewall rule (obviously). It can be done with tcp/135 or an IP any between hosts. Use the command:
show run policy map to verify the policy map.

Cisco ASA show CLI passwords

What’s the command so that you can see hashed passwords on the Cisco ASA CLI?

Answer:
ciscoasa(config)# more system:running-config

Very handy for when you need to copy passwords from one device to the other.