Getting started with Network Automation is all about use cases. It's hard to boil the ocean, take an existing traditional network and start managing it through Ansible using Infrastructure-as-Code, especially if you haven't done it before. You have to start somewhere small, where the risks are low but the benefits are clear. Find the right use case, and go for it.

The other day, I ran into just such a use case. Our project to scale Cisco AnyConnect VPN  for the entire company instead of just a handful of people just went live. But with so many people working from home now, we saw a huge uptick in the use of Teams and Skype for Business. So we got the request to see if we could optimize this traffic by excluding it from the full tunnel VPN.  Now we can, of course, look up the Office 365 endpoints in the Microsoft documentation, write a split-tunnel ACL for these addresses, and configure this in the ASAs; but we run into two problems here:

  1. Microsoft can add and remove IP addresses to this list at any time. They do publish any changes to an RSS feed, but we have to figure out a way to monitor this and turn it into service desk tickets. It would be nice to be able to do this dynamically.
  2. With our new scale, we are no longer running a pair of ASAs, but a few dozen of them, all running the same VPN configuration. I would like to avoid having to log into each one individually and have a way to ensure that all of them are using the same config.

A clear use case for trying some automation. The first step (and subject of this post) is getting a list of Office365 Endpoint IP-addresses in a format we can use (and learn a bit of Python along the way). In the next part we'll worry about getting that config to the ASAs.

Office 365 Endpoints

First order of business is getting a machine-readable list of IP addresses from Office365, which means querying a REST API to get a JSON object containing all of them. Microsoft provides such an API, along with some instructions on how to use it.

The simple version is: use this URL to get a full list of all Office365 endpoint IPs and URLs:

https://endpoints.office.com/endpoints/worldwide?clientrequestid=b10c5ed1-bad1-445f-b386-b919946339a7.

You can click on this link to get a JSON object right in your browser; but to parse it and make it repeatable, we will need to write a script to do this. As network engineer, my go-to language for this is Python.

Preferred programming languages from the NetDevOps Survey 2019

generating a guid

If you look closely at the endpoints URL, you'll notice that there's an parameter at the end: ClientRequestId=. This parameter is mandatory, the URL doesn't work without one. Now of course we can generate a GUID once and hardcode it in our code, but it's always a good idea to avoid hardcoding and I prefer to generate one at runtime.

Luckily, there's a built-in Python library to do exactly that: UUID.  And it's quite easy to use:

import uuid

def generate_guid():
    """Randomly generate new GUID at runtime."""
    uid = str(uuid.uuid4())
    return uid

guid = generate_guid()

Okay, creating a function for one line of code is overkill, but it's a good habit to get into. It makes for nicely modular, readable code. And for the other parts of our script it'll make more sense.

filtering the results

Now we have a choice to make. We can either grab the entire list and filter later on in our code to what we want, or filter the request to receive only the items we need. For this specific use case, both options are valid; but as a good practice it's best to filter on the request, to reduce network traffic and speed up the API call.

The documentation at microsoft.com lists a few parameters we can use to filter the results:

Parameters for the endpoints web method are:

  • ServiceAreas=<Common | Exchange | SharePoint | Skype> — A comma-separated list of service areas. Valid items are Common, Exchange, SharePoint, and Skype. Because Common service area items are a prerequisite for all other service areas, the web service always includes them. If you do not include this parameter, all service areas are returned.
  • TenantName=<tenant_name> — Your Office 365 tenant name. The web service takes your provided name and inserts it in parts of URLs that include the tenant name. If you don't provide a tenant name, those parts of URLs have the wildcard character (*).
  • NoIPv6=<true | false> — Set the value to true to exclude IPv6 addresses from the output if you don't use IPv6 in your network.
  • Instance=<Worldwide | China | Germany | USGovDoD | USGovGCCHigh> — This required parameter specifies the instance from which to return the endpoints. Valid instances are: Worldwide, China, Germany, USGovDoD, and USGovGCCHigh.

For my use case, I need only Skype/Teams endpoints, and only IPv4 addresses. So let's write a function to assemble a request URL to return only the data we need.

def assemble_url(services, ipv6):
    """Assemble Office365 enpoint query URL for selected services."""
    base_url = "https://endpoints.office.com/endpoints/worldwide"
    guid = generate_guid()
    service_list = ",".join(services)
    no_ipv6 = str(not ipv6)

    # assemble url
    full_url = f"{base_url}?clientrequestid={guid}&ServiceAreas={service_list}&NoIPV6={no_ipv6}"
    return full_url

SERVICES = [ "Skype" ]  # allowed values: <Common | Exchange | SharePoint | Skype>
IPV6 = False  # allowed values: <True | False> 
request_url = assemble_url(SERVICES, IPV6)

querying the REST API

We can open the request URL in a browser, or if we were using shell scripting we could use a tool like curl or wget. In Python, the de facto standard for making REST API calls is the requests library. This library is not installed by default, don't forget to install it using pip:

pip install requests

The nice thing about requests is that it not only returns the JSON object we need, but also allows us to do things like check the status code to see if the request was successful. We wouldn't want to populate our ASA with an empty ACL just because the REST API returned 503 Service Temporarily Unavailable. Another nice thing is that requests can automatically parse JSON to a Python dictionary by calling the .json() method. We can create a nice function for this, making use of the assemble_url function we wrote earlier.

import requests

def get_o365_ips(services, ipv6):
    """Fetch Office365 IP-addresses for selected services."""
    request_url = assemble_url(services, ipv6)
    # fetch IPs
    response = requests.get(request_url)
    if response.status_code == 200:
        content = response.json()
    else:
        raise Exception("IP fetch failed")
    return content

response = get_o365_ips(services=["Skype"], ipv6=False)

Turning it into something useful

We now have a Python object containing only the IPv4 addresses for the Office365-services we are interested in (Skype).  If you take a look at the results (perhaps by opening the assembled URL in your browser), you'll see that it's not a simple list of IP addresses, but also contains a lot of extraneous information:

[
  {
    "id": 11,
    "serviceArea": "Skype",
    "serviceAreaDisplayName": "Skype for Business Online and Microsoft Teams",
    "ips": [
      "13.107.64.0/18",
      "52.112.0.0/14",
      "52.120.0.0/14"
    ],
    "udpPorts": "3478,3479,3480,3481",
    "expressRoute": true,
    "category": "Optimize",
    "required": true
  },
  {
    "id": 12,
    "serviceArea": "Skype",
    "serviceAreaDisplayName": "Skype for Business Online and Microsoft Teams",
    "urls": [
      "*.lync.com",
      "*.teams.microsoft.com",
      "teams.microsoft.com"
    ],
    "ips": [
      "13.70.151.216/32",
      "13.71.127.197/32",
      "13.72.245.115/32",
      "13.73.1.120/32",
...

We need to extract only the IP addresses, and present them in a way we can use in our ASAs.

extract IP prefixes

If you look closer at the JSON response, you'll see that it's a list of dictionaries containing several keys. We are only interested in the key "ips". We can easily extract them using a for loop. We'll add some additional logic to eliminate duplicate entries in our list while we're at it.

def parse_response(response):
    """Parse Office365 Endpoints API results to a list of IP addresses."""
    ip_list = list()
    for entry in response:
        if 'ips' in entry:  # ignore everything that isn't an IP
            ip_list += entry['ips']
    clean_list = (dict.fromkeys(ip_list))  # automatically remove duplicates
    return clean_list

prefix_list = parse_response(response)

convert prefix to network + netmask

We're one step further, we now have a clean list of network prefixes in the form '192.0.2.0/24'. Unfortunately, the ASA doesn't handle network prefixes well, so we'll need to split it into a network address and netmask. We could do this by splitting the string on the '/' and doing some binary math on the netmask... and while that's an excellent exercise for honing or coding skills, as usual with Python there's a library that can handle all the messy details for us: ipaddress.

Using this module, we can create a function that takes a prefix and returns address and netmask parts separately.

import ipaddress

def prefix_to_network(prefix):
    """Convert an IP prefix to an IP-address and network mask."""
    ipaddr = ipaddress.ip_interface(prefix)  # turn into ipaddress object
    address = ipaddr.ip
    mask = ipaddr.netmask
    return address, mask

test_ip = "192.0.2.0/24"
print(prefix_to_network(test_ip))

Now we're nearly there. The final piece is to take our list of prefixes (split into network and netmask), and turn it into something we can feed the ASA. For now we'll just print it to texts, so we can test it manually. We'll create an object-group that we can reference in an ACL, which should look something like this:

hostname (config)# object-group network admins
hostname (config-protocol)# description Administrator Addresses
hostname (config-protocol)# network-object host 10.2.2.4
hostname (config-protocol)# network-object host 10.2.2.78
hostname (config-protocol)# network-object host 10.2.2.34

We'll loop over all prefixes in our prefix list, split it using our prefix_to_network function, and print it as network-object:

def ip_list_to_object_group(ip_list):
    """Translate IP prefix list to ASA network object-group."""
    object_group = list()
    object_group.append("object-group network o365_addresses")
    object_group.append("  description Office 365 endpoint addresses")
    
    for ip in ip_list:
        network, mask = prefix_to_network(ip)
        object_entry = f"  network-object {network} {mask}"
        object_group.append(object_entry)
    return object_group

print("\n".join(ip_list_to_object_group(prefix_list)))

Now we have something we can use. If we put all the functions we wrote into a script, we can run it and output ASA CLI code for an object-group:

[email protected]:~/asav_office365$ python get_o365_ips.py
object-group network o365_addresses
  description Office 365 endpoint addresses
  network-object 13.107.64.0 255.255.192.0
  network-object 52.112.0.0 255.252.0.0
  network-object 52.120.0.0 255.252.0.0
  network-object 13.70.151.216 255.255.255.255
  network-object 13.71.127.197 255.255.255.255
<...> etcetera

What's next?

In the next part of this series, we'll see how we can push this configuration to all of our ASAs. For now, if you want to see the full script: I've put it on GitHub.