01-04-2014, 04:53 PM
Disturbance Data Collection Solution Using SYNC 3000 and SYNC 4000
[attachment=60752]
BUSINESS CASE
The Client was one of the largest transmission utility company in the world, with more than 95,000km Transmission
network and more than 200 EHV / HVDC substations, planned to set up a Remote Access and Automatic Fault Analysis
System from each of its substation spread across an area of 2.5Mn sq.kms. The main functionalities of the system was
to include the automatic remote collection of the Disturbance Recording (DR) files, as well as Event Logs (EL) from the
various devices / IEDs in the substations, and subsequent download to a central National Transmission Management
Center, through nice Regional Asset Management Centers spread across the country. The Utility was able to remotely
view, configure devices, collect disturbance and event data – hence, making it possible for the substations to be un-
manned and operated / maintained remotely. The project involved almost all of the client's 200EHV substations, and
more than 3000 relays / IEDs, and was the largest such asset monitoring project executed in the country's transmission
network.
Apart from the scale, the complexity of the project was magnified as the data was to be collected from various IED
manufacturer, hence was necessary to have a large protocol conversion capability at the collector end. The absence of
Ethernet ports in all relays also made it necessary to use terminal servers, for enabling remote configuration for all the
existing devices in the substations.
SOLUTION
Kalkitech proposed a de-centralized architecture where a redundant Kalkitech SYNC 3000 was provided in every
substation that acted as Substation Disturbance Record (DR) collector. The disturbance data was captured from the
IEDs using proprietary/standard protocols and then converted to COMTRADE (refer last page on more information
regarding COMTRADE file format) files by SYNC 3000 at the substation and sent to Kalkitech's SYNC 4000 server at
National Asset Management center, via a secure communication channel. COMTRADE files in SYNC 4000 at the
management center were mirrored to SYNC 4000 at a back-up site when new DR files were generated.
The SYNC 4000 also acted as a server to numerous enterprise applications and clients, including the Automatic Fault
Analysis System, SCADA and DR Monitoring and Analysis Applications. Hence the SYNC 4000 also had various
adaptors that facilitated data transfer, including ICCP, ODBC, SMB.CIFS and proprietary adaptors.
KALKITECH SYNC 3000 Data Concentrator was widely used in substation data concentrator and protocol converter,
which was designed for gathering of real-time data and files from IEDs, Event Loggers installed in transmission and
distribution substations. SYNC 3000 communicated with substation devices and managed the collection, processing and
presentation relating to faults and disturbances at each substation.
DATA ACQUISITION USING SYNC 3000 and SYNC 2000:
The SYNC 3000 station data concentrator was configured in a Primary - Standby modes such that the complete
functionality of the RAS was available even in case of failure of one. The data (disturbance records / files / logs /
database changes /updates) were synchronized between the two data concentrators by the Hot Standby (HSB) module
in SYNC 3000. SYNC 3000 also had IEC 61850-3 substation ruggedness compliance with 6 Ethernet and 12 serials
ports, for direct connection of Relays and Event Loggers.
For substation where more than 12 serial ports were needed for DR collection, Kalkitech's SYNC 2000 Terminal server
with 6 serial ports was used such that not more than 5 relays were connected on one serial loop. SYNC 2000 terminal
servers were used in a pass through mode while DR collection drivers and conversion to COMTRADE format was
executed in SYNC 3000. SYNC 3000 Data Concentrator (DC) also had two or six Ethernet port option, connected to the
Station Ethernet network, configuration LAN and SAS LAN fetched data from IEDs connected in substation.
For field devices with RS232 serial ports, a Fall Back Switch was used to provide connectivity of the serial link to primary
and standby SYNC 3000 data concentrators. For field devices with serial ports that were multi-dropped (such as SPA,
Courier) the RS485 loop were connected to both active and stand-by SYNC 3000 units.
SYNC 4000 AS CONTROL CENTER SERVER
Kalkitech's SYNC 4000, installed at the Asset Management center, managed secure communication to Station DC and
provides remote access for configuration. It also supported an interface to ISR and SCADA system and accepted DR
collection trigger as well as, sent stored event data from Event loggers in ISR. SYNC 4000 hardware proposed with Dual
LAN, 8 GB RAM expandable to 32GB RAM, 320 GB HDD (expandable to 640GB), with RAID 5 storage for long term
storage (with each Server).
SYNC 4000's DR collection system user interface came with functionalities to perform configuration change, view status
of DR collection, view alarms or events in the communication system. Option was also provided for manual DR collection
triggered to initiate DR transfer from the substations to the central asset management center.
COMTRADE File transfer from SYNC 3000 DC to SYNC 4000 at the Asset Management Centre was using Secure File
Transfer Protocol (SFTP), with SYNC 3000 acting SFTP server whereas SYNC 4000 was SFTP client. The client- server
communication was encrypted using SSL. The data exchange between SYNC 3000 and SYNC 4000 was compliant to
NERC CIP requirements. SYNC 4000 logs all user initiated activities performed along with necessary information such
as time-stamp and user/administrator comments needed as part of maintaining audit trails. It also created communication
logs to identify communication errors.
KEY FEATURES
1. Enhanced Security:
- NERC/CIP compliant security
- User management in Configuration utility to prevent unauthorized configuration
- End to end SSL encrypted VPN to avoid tampering of data transferred in the network
Integration of Protection and Control IEDs in the substation was carried out in a secured manner in compliance with
relevant NERC CIP Cyber Security standards particularly CIP-005-1 (Electronic Security).
Remote Access to the IEDs in the substation was restricted through a single point by the Substation Firewall provided
by the System Integrator; it created an electronic perimeter that protected all the electronic devices e.g. IEDs/ Relay/
DR /EL from unauthorized access. It also performed authentication with user name and password authorization by
assigning users to groups with well defined privileges, Encrypted all pass-thru connections that spanned the WAN
connection.
[attachment=60752]
BUSINESS CASE
The Client was one of the largest transmission utility company in the world, with more than 95,000km Transmission
network and more than 200 EHV / HVDC substations, planned to set up a Remote Access and Automatic Fault Analysis
System from each of its substation spread across an area of 2.5Mn sq.kms. The main functionalities of the system was
to include the automatic remote collection of the Disturbance Recording (DR) files, as well as Event Logs (EL) from the
various devices / IEDs in the substations, and subsequent download to a central National Transmission Management
Center, through nice Regional Asset Management Centers spread across the country. The Utility was able to remotely
view, configure devices, collect disturbance and event data – hence, making it possible for the substations to be un-
manned and operated / maintained remotely. The project involved almost all of the client's 200EHV substations, and
more than 3000 relays / IEDs, and was the largest such asset monitoring project executed in the country's transmission
network.
Apart from the scale, the complexity of the project was magnified as the data was to be collected from various IED
manufacturer, hence was necessary to have a large protocol conversion capability at the collector end. The absence of
Ethernet ports in all relays also made it necessary to use terminal servers, for enabling remote configuration for all the
existing devices in the substations.
SOLUTION
Kalkitech proposed a de-centralized architecture where a redundant Kalkitech SYNC 3000 was provided in every
substation that acted as Substation Disturbance Record (DR) collector. The disturbance data was captured from the
IEDs using proprietary/standard protocols and then converted to COMTRADE (refer last page on more information
regarding COMTRADE file format) files by SYNC 3000 at the substation and sent to Kalkitech's SYNC 4000 server at
National Asset Management center, via a secure communication channel. COMTRADE files in SYNC 4000 at the
management center were mirrored to SYNC 4000 at a back-up site when new DR files were generated.
The SYNC 4000 also acted as a server to numerous enterprise applications and clients, including the Automatic Fault
Analysis System, SCADA and DR Monitoring and Analysis Applications. Hence the SYNC 4000 also had various
adaptors that facilitated data transfer, including ICCP, ODBC, SMB.CIFS and proprietary adaptors.
KALKITECH SYNC 3000 Data Concentrator was widely used in substation data concentrator and protocol converter,
which was designed for gathering of real-time data and files from IEDs, Event Loggers installed in transmission and
distribution substations. SYNC 3000 communicated with substation devices and managed the collection, processing and
presentation relating to faults and disturbances at each substation.
DATA ACQUISITION USING SYNC 3000 and SYNC 2000:
The SYNC 3000 station data concentrator was configured in a Primary - Standby modes such that the complete
functionality of the RAS was available even in case of failure of one. The data (disturbance records / files / logs /
database changes /updates) were synchronized between the two data concentrators by the Hot Standby (HSB) module
in SYNC 3000. SYNC 3000 also had IEC 61850-3 substation ruggedness compliance with 6 Ethernet and 12 serials
ports, for direct connection of Relays and Event Loggers.
For substation where more than 12 serial ports were needed for DR collection, Kalkitech's SYNC 2000 Terminal server
with 6 serial ports was used such that not more than 5 relays were connected on one serial loop. SYNC 2000 terminal
servers were used in a pass through mode while DR collection drivers and conversion to COMTRADE format was
executed in SYNC 3000. SYNC 3000 Data Concentrator (DC) also had two or six Ethernet port option, connected to the
Station Ethernet network, configuration LAN and SAS LAN fetched data from IEDs connected in substation.
For field devices with RS232 serial ports, a Fall Back Switch was used to provide connectivity of the serial link to primary
and standby SYNC 3000 data concentrators. For field devices with serial ports that were multi-dropped (such as SPA,
Courier) the RS485 loop were connected to both active and stand-by SYNC 3000 units.
SYNC 4000 AS CONTROL CENTER SERVER
Kalkitech's SYNC 4000, installed at the Asset Management center, managed secure communication to Station DC and
provides remote access for configuration. It also supported an interface to ISR and SCADA system and accepted DR
collection trigger as well as, sent stored event data from Event loggers in ISR. SYNC 4000 hardware proposed with Dual
LAN, 8 GB RAM expandable to 32GB RAM, 320 GB HDD (expandable to 640GB), with RAID 5 storage for long term
storage (with each Server).
SYNC 4000's DR collection system user interface came with functionalities to perform configuration change, view status
of DR collection, view alarms or events in the communication system. Option was also provided for manual DR collection
triggered to initiate DR transfer from the substations to the central asset management center.
COMTRADE File transfer from SYNC 3000 DC to SYNC 4000 at the Asset Management Centre was using Secure File
Transfer Protocol (SFTP), with SYNC 3000 acting SFTP server whereas SYNC 4000 was SFTP client. The client- server
communication was encrypted using SSL. The data exchange between SYNC 3000 and SYNC 4000 was compliant to
NERC CIP requirements. SYNC 4000 logs all user initiated activities performed along with necessary information such
as time-stamp and user/administrator comments needed as part of maintaining audit trails. It also created communication
logs to identify communication errors.
KEY FEATURES
1. Enhanced Security:
- NERC/CIP compliant security
- User management in Configuration utility to prevent unauthorized configuration
- End to end SSL encrypted VPN to avoid tampering of data transferred in the network
Integration of Protection and Control IEDs in the substation was carried out in a secured manner in compliance with
relevant NERC CIP Cyber Security standards particularly CIP-005-1 (Electronic Security).
Remote Access to the IEDs in the substation was restricted through a single point by the Substation Firewall provided
by the System Integrator; it created an electronic perimeter that protected all the electronic devices e.g. IEDs/ Relay/
DR /EL from unauthorized access. It also performed authentication with user name and password authorization by
assigning users to groups with well defined privileges, Encrypted all pass-thru connections that spanned the WAN
connection.