Building a Data Center to Accomodate Network Monitoring

by Karl Tatgenhorst on February 15, 2011

Designing a network right is an important step for securityNetwork security is an afterthought or an add on. How many times have you heard that? I’m going to frame this article (possibly series) for someone who is building a new data center. I won’t get into the physical security aspect of it, because that will differ from site to site but that doesn’t negate the importance of it. Most pieces of network architecture are very vulnerable to physical attacks so be sure to properly plan that based on your specific threat model.

The normal operation of a networked computer has specific behaviors which, over time we can learn to recognize. With proper planning, we can build our data center to facilitate the generation of and utilization of logs pertaining to the behavior of the machines, Using these logs and our own knowledge if proper patterns. An example of aberrant behavior that would have alerted us to an attack would be any one of a number of worms from the late 90s. These worms were hard coded to use specific DNS servers instead of using the compromised machines resolver and thus the native DNS server for the host network. This would cause anyone monitoring the network to notice sudden DNS calls to other DNS servers. To allow this behavior to be caught, two things would need to be present on our network. First, the network would need to have assigned values for necessary infrastructure such as DNS (we couldn’t identify anomalous traffic if we don’t define normal traffic). Second, the network traffic has to be capable of being logged or analyzed in real time.
There are a variety of ways for traffic to be logged or analyzed. One method is to configure a span port on your switch, which will mirror all of the traffic for the network within that switch. This mirrored traffic can either be sent to something which can record or summarize it, or it can be sent to some type of real time sensor. Summarized traffic from a switch or router is usually referred to as flow data. Net flows are a type of internal accounting that routers used to communicate between each other regarding traffic. A netflow would contain at a minimum: source ip, destination ip, timestamp source interface and destination interface as well source port and destination port. In addition to being created by a device external to the switch or router, Cisco layer 3 devices (including some 3xxx series switches) can send netflows over UDP to a collector. Having the device send flows is especially good if the collector is not in close proximity to the switch. Netflow auditing adds roughly 0.8% to the overall traffic on a network and is thus unlikely to cause any congestion issues.
Now we have a way of identifying anomalous traffic on the network, however we don’t have a way to tie any traffic to a machine on the network. This is yet another area where physical design of your network should have attention paid to it. Each physical jack in the areas that your data center serves should be clearly labelled with some type of code, that code should exist as a pair with the switch and port number that it is connected to. Using SNMP traps we can get the mac address of any machine that connects to the network as well as what switch/port number it is connected to. Using the network jack/switch-port key pairs we can then tell what mac address is on that port. We can use RARP to get a mac address from an ip address or better still if we control the DHCP server we can query the DHCP logs and find it (if it used DHCP).
That gives us two major design features to make our data center network security friendly, we should also design our network to mitigate threats where possible. A threat model has the following components: threat, vulnerability, asset value and risk. Risk is computed by assigning a numeric value to threat, vulnerability and asset value then multiplying the 3 numbers. Reducing any of those values will reduce the overall threat.
Creating VLANs (networks separated by a layer 3 gateway) for different asset types and their consumers will mitigate the amount of access an intruder would have if he were to gain access to the network. Additionally, Unix machines could be multi homed with one nic being on a public network and only critical applications are available on it and the other would be a management network and all management applications would use that network. Remember when planning out VLANs they must either all exist on a switch which can perform layer 3 tasks (routing) or you will need a router with an interface for each VLAN.
Using these principles is a good start to building a network which can be easily monitored and also has some security built into it. What other principles could be added to this?

About the author

Karl Tatgenhorst wrote 31 articles on this blog.

  • Pingback: Tweets that mention Building a Data Center to Accomodate Network Monitoring — Karl Tatgenhorst --

  • Pingback: Using SNMP Traps To Provide Physical Visibility To Your User Base()

  • before i go to sleep summary

    Hi there! I simply want to give an enormous thumbs up for the nice
    information you could have here on this post.
    I shall be coming again to your blog for extra soon.

  • thailand plastic surgery price list

    Hey! I simply wish to give a huge thumbs up for the good data
    you have got here on this post. I can be coming again to your weblog for extra soon.

  • future futureless futures futurism futurisms futurist futuristic futuristically futurists futurities futurity futurologist futurologists futurology fuze fuzed fuzee fuzees fuzes fuzil fuzils fuzing fuzz fuzzed fuzzes fuzzier fuzziest fuzzily fuzziness fuz

    Hiya! I just want to give a huge thumbs up for the
    good data you have got right here on this post.
    I shall be coming back to your weblog for more soon.

Previous post:

Next post: