Challenges in the capture and dissemination of measurements from high-speed networks

Abstract: The production of a large-scale monitoring system for a high-speed network leads to a number of challenges. These challenges are not purely techinical but also socio-political and legal. The number of stakeholders in a such a monitoring activity is large including the network operators, the users, the equipment manufacturers and of course the monitoring researchers. The MASTS project (Measurement at All Scales in Time and Space) was created to instrument the high-speed JANET Lightpath network, and has been extended to incorporate other paths supported by JANET(UK). Challenges the project has faced have included: simple access to the network; legal issues involved in the storage and dissemination of the captured information, which may be personal; the volume of data captured and the rate at which this data appears at store. To this end the MASTS system will have established four monitoring points each capturing packets on a high speed link. Traffic header data will be continuously collected, anonymised, indexed, stored and made available to the research community. A legal framework for the capture and storage of network measurement data has been developed which allows the anonymised IP traces to be used for research pur poses.