SLAC, through the DOE funded DWMI initiative, currently configures the intrusiveness of active network measurement (e.g. iperf, ping, traceroute, pipechar etc.) based on known network capabilities, and use passive means such as SNMP, Netflow and host level solutions such as Web100 extensively.
We have studied and evaluated the most effective network monitoring tools for each network and believe that as end-host performance and network utilisation increases, it will become essential to be able to capture information from reliable passive means such a real application transfers and directly from routers and switched.

As it will be important to instrument the network with the relevant sensors to provide network measurement and monitoring capabilities, we will help to evaluate and federate the various network monitoring solutions available.
We will also provide assistance to deploy relevant sensors at important institution centers to aid the measurement resolution of network elements.

An important issue to consider with the passive gathering of network performance information from network elements is that the physical and logical components are typically owned and managed by different network operators.
Projects such as perfSONAR [perfSONAR] and the Abilene Measurement Infrastructure (AMI) [AMI], of which both SLAC are involved with, aim to provide the reporting of SNMP information from network elements in unified and standardized way.
Using the GGF NMWG schema (which SLAC helped to develop), the flexible XML specification, coupled with service orientated architecture design shall provide both applications and end-users very detailed network performance related information.

However, such projects only provides a broad picture of the actual network usage's for the entire network element/link and as such, other techniques must be used to complement this information in order to be useful for applications.
Through utilizing both NetFlow data (if and when available from the various intermediatory autonomous systems), active end-to-end tests, and direct passive monitoring of real applications, we plan to provide a rich set of data that will help to characterize and optimize application level performance.
An issue with such information is that often the datasets are often very large (typically several GBs per day), and will often require anomolizing to vet security concerns of gathering private information.
We currently have methods of summarising such data and efficiently processing such data dynamically which we use for local NetFlow records.

Through extending DWMI initiative to fully support an end-to-end monitoring solution involving both end-systems and intermediate routers and switches, a full depiction of the performance bottlenecks and performance trends can be determined.

As a direct consequence of being able to abstract network performance metrics from its implementation, we wish to develop additional services that will provide valuable information to users.
By working closely with real SciDAC application groups to determine their network requirements, and by providing federated and standardised web-services and leveraging (and possibly adapting) the NMWG [NMWG] schema we aim to provide a API from which network monitoring information can be easily provided to both network users (network managers and end-users) and for application control and or steering.
We believe three such features of the API that will be useful for applications is that of bottleneck location identification, anomalous event detection, network performance forecasting.
These are discussed as follows.

Anomalous Event Detection: 
The mining of such network data will provide a rich and extensive framework from which network problems and issues can be determined.
However, the process typically involves labour intensive manual diagnosis and cross correlation to pinpoint and identify such anomalies.
Therefore there is a major advantage in being able to automate such problem determination through mathematical and systems based design.
This event detection can then be applied to determine potential and real problems experienced on the network and alerts provided to relevant parties such that network issues can be resolved prior to end-users experiencing such problems.
We are currently mining IEPM-BW network information to test the validity of different algorithms when applied to different networking metrics and network scenarios. 
We are also working closely with institutions such as Internet2, ESnet and Geant to help validate our findings and to help automatically pinpoint the exact cause of network performance degradation's; whether it be on the end-hosts or across the wide area network.
The implementation of such notifications have vastly improved the ability to both determine the occurrence of such problems (from days to hours).
However, much more research needs to be conducted in order to determine the best algorithms for such event detection.

Bottleneck Location Identification: 
The facility to be able to determine whether there is a problem on the network will be extended to provide details of the location(s) that represent this 'bottleneck'.
Bottleneck detection will become important in the future as network resources become more competitive as end-host link speeds increase.
SLAC are working closely with projects such as OSCARS [OSCARS], Terapaths [TERAPATHS] and CHEETAH [CHEETAH] that will provide advanced network resource reservation end-to-end.
However, such 'blind' reservation of network resources will require not only the successful monitoring of application traffic, but also the need to isolate problem regions whereby static or dynamic (re)configuration can take place to alleviate network problems.
We are currently exploring the application of specific network tools (such as PathNeck [PATHNECK]) and using federated services such as perfSONAR to develop mathematical and systematic isolation techniques to discover bottlenecks in the real Internet.
We are currently working with the Internet2 piPES [PIPES] program and wish to extend our involvement in the project by utilizing our expertise in network monitoring and event detection.

Network Performance Forecasting:
In parallel with the development and deployment of network sensors, data from IEPM-BW and AMI measurement servers located in real production networks will be used to evaluate and develop short and long-term (hours to days) forecasting techniques for predicting bottleneck magnitude and location.
The forecasts will take into account seasonal patterns and long term trends in the data and will build on the existing work by SLAC in this area [SLACFORECASTING].
These forecasts, including confidence levels, will form the foundation of higher level services such as application network provisioning.
The forecasting will initially be based on the Holt-Winters [HOLTWINTERS] triple Exponential Weighted Moving Averages (EWMA) technique for time series that exhibit short term variations, long term trends and seasonal changes.
This technique will be applied to the various time-series of active and passive measurements.
We will also evaluate other techniques such as Principal Component Analysis, wavelets, and/or the use of neural networks, etc.