Control Room Monitor Review
Last Revised: 05/15/19
Monitor Name How to Start and Exit Key Elements to Monitor Symptom: Response:
Stats Monitors:
Database Loaders

Processes read data files and load the records into the databases.

Monitor shows processing stats of each loader.

To Start:
Use option from PROD MENU:
 DB Loader Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Rejects,
- Percent Complete,
- Records remaining,
- Insert Rate,
- Minutes to Finish
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Reject File Size column shows positive value (non-zero). Rejects have occurred.
Process rejects per procedures.
Percent Complete and Records Remaining columns showing non-zero values and not reducing as expected. Insert Rate is zero or lower than expected rate. Process is either down or hung, or database is not responding.
1) Check status of process
2) If process is up, check CPU and memory utilization to see if process is hung.
3) If process is up and not hung, call Database Technologies.
Database
Copy Queue Monitor

Script to copy records from DB1 to DB2, if data is not dually loaded through database loaders.

Monitor shows processing queue of script.
To Start:
Use option from PROD MENU:
 DB Loader Monitoring Menu

To Exit:
Close Window
- Date/Time
- "Count Is" Values
ERROR: invalid username/pasword Process is not being run from ops$monitor account. Run from ops$monitor account.
Count is values are non-zero, and not reducing as expected.
Data is not being copied as expected.  Call Database Technologies.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from Site1 to Site2.

Monitor shows connection status between Site 1 fix engine and firm, or of fix engine and Site 1 application.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
Site 2 may not be processing as expected.
1) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 App Processing Stats

Processes facilitate communications from Site 1 to Site 2.

Monitor shows processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- InCount,
- OutCount,
- LastInTime,
- LastOutTime
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
No data is displayed. No data has been generated to any Site 2. 
1) Check APP1 log files. If all sizes are zero, no traffic has been generated.
2) Test sending order to Site 2.
InCount does not match OutCount. Site 2 may not be processing as expected.
1) Check APP1 message files to confirm inbound messages match outbound messages.
2) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate communications from Site 1 to Site 2.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from Site 1 to Site 2.

Monitor shows IPC channel connectivity status to APP2 processes.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from Site 1 to Site 2.

Monitor shows IPC channel connectivity status to APP2 processes.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2  Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 1 to SITE 2 Destinations.

Monitor shows connection status between SITE 1 fix engine and SITE 2.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
Site 2 may not be processing as expected.
1) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 2 directly to SITE 1.
(Direct access Protocol Buffer)

Monitor shows connection status between SITE 2 and SITE 1, or connection status and APP2, by topic.

To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
Status is Inactive Third Party communications has been disabled between the Site 1 and the App2.
1) Check status of process
2) If process is up, call Production Support. 
Write Queue is non-zero values and not reducing as expected.
Site 2 may not be processing as expected.
1) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 App Processing Stats

Processes facilitate communications from SITE 2 directly to SITE 1.
(Direct access Protocol Buffer)

Monitor shows processing statistics of application.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- InCount,
- OutCount,
- LastInTime,
- LastOutTime
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
No data is displayed. No data has been generated by order sending firm. 
1) Check APP1 message files. If all sizes are zero, no traffic has been generated.
2) Call firm if necessary.
InCount is not less than, or equal to OutCount. Site 2 may not be receiving all matching engine responses expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call SITE 2 if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 1 (APP1 via APP2 and APP3 processes) to SITE 2 Destinations.

Monitor shows connection status between APP1 and firm as well as processing stats between APP2 (msgin) and APP2 as well as site 2 (msgout).
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- fixConnect,
- MsgIn,
- MsgOut
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
FixConnect is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
MsgIn values are not equal to or greater than MsgOut values.

Depending on APP1 configuration, outbound messaging may be filtered such that not all inbound messages received by APP2 will be forwarded to firm.
Site 2 communications may not be as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Site 2 if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 1 to SITE 2 Destinations.

Monitor shows connection status between APP1 and SITE 2 as well as processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- nInNum,
- nOutNum
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
InCount does not match OutCount. We may not be processing as expected.
1) Check APP1 message files to confirm inbound messages match outbound messages.
2) Work with user community and SITE 2 if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 2 to SITE 1 for special routing (via APP2 to APP3 , APP4 or APP5).

Monitor shows connection status between SITE 1 and SITE 2, or of APP1 and APP2.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
Site 2 may not be processing as expected.
1) Call Site 2 and work with Technical Services if necessary.


Stats Monitors:
APP1 App Processing Stats

Processes facilitate communications from SITE 2 to SITE 1 for special routing (via APP2 to APP3, APP4 or APP5).

Monitor shows processing statistics between SITE 1 and SITE 2.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- InCount,
- OutCount,
- LastInTime,
- LastOutTime
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
No data is displayed. No data has been generated between SITE 1 and SITE 2.
 
1) Check APP1 message files. If all sizes are zero, no traffic has been generated.
2) Test sending order to test APP1.
Aggregate InCount (by service name) is not less than, or equal to aggregate (by service name) OutCount.

NOTE: LastOutTime (by service name) should also be later than LastInTime (by service name)
Site 2 may not be processing as expected.
1) Check APP1 files to confirm inbound messages match outbound messages.
2) Call SITE 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate communications from SITE 2 to SITE 1 for special routing (via APP2 to APP3, APP4 or APP5).

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero. No messages have been sent/received since that monitor has been started.
1) Check APP2 log files.
2) Call Site 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 to SITE 1 for special routing (via APP2 to APP3, APP4 or APP5).

Monitor shows IPC channel connectivity status to APP2 processes.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 to SITE 1 for special routing (via APP2 to APP3, APP4 or APP5).

Monitor shows IPC channel connectivity status to APP2 processes.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate communications from SITE 1 processes (such as APP2 and APP3) and APP4.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call SITE 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 1 to SITE 2 via APP1.

Monitor shows connection status between APP1 and SITE 2 as well as processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- SzConnStat,
- nInSeqNum,
- nOutSeqNum
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
szConnStat is Disconnected or Open Site 2 is not connected.
1) Use Control Utility ORS Service Controls to Open Channels.
2) Call Production Control if needed.
nInSeqNum is not greater than, or equal to nOutSeqNum.
We may not be processing as expected.
1) Check APP1 message files to confirm inbound messages match outbound messages.
2) Work with SITE 2 and Tech Services if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 2 and APP1 via APP2.

Monitor shows connection status between APP1 and APP2 as well as processing statistics.
To Start:
Use option from PROD MENU:
Firmswitch Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- OutMsgs,
- InMsgs
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
InMsgs value is not relatively close to, or far greater than OutMsgs value.

Generally, all messages (in and out) are heartbeats traded between sides with the exception of drop copies received from the firms.

We may not be processing as expected.
1) Check APP1 message files to confirm inbound messages match outbound messages.
2) Work with SITE 1 and SITE 2 if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from SITE 1 applications and APP1 Database Loader.

Monitor shows connection status between APP1 and APP2 applications writing to it.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Processes are not connected.
1) Check processes status.
2) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
Data may not be loading into database as expected.
1) Call production support if necessary.
InCnt values are not identical between "sister" market data processes.  IE., A and B series processes should be identical.
Data may not be loading into database as expected.
1) Call production support if necessary.
Stats Monitors: from SITE 2 (DC1 and DC2)

Processes facilitate communications from SITE 2 and SITE 3 to APP3 and APP2 via APP1.

Monitor shows connection status between APP1 process and SITE 2 primary/alternate channels as well as processing statistics of each process.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Primary Connection,
- Alt Connection,
- Primary Rate,
- Alt Rate,
- Prim Total Msgs,
- Alt total msgs
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Primary Channel and/or Alt Channel are not ENABLED. Processes are not reading multicast SITE 2 data.
1) Use Control Utility APP1 Service Controls to control multicast readers.
2) Call Production Control if needed.
Primary Rate and/or Alt Rate show sustained rate of zero during trading hours. Multicast feed is not being processed as expected.
1) Work with SITE 2 and Technical Services if necessary.
Primary Total Msgs and Alt Total Msgs are not showing (relatively) the same numbers of messages processed per process. 
Multicast feed is not being processed as expected.
1) Work with SITE 2  and Technical Services if necessary.
Stats Monitors:
APP1 to APP2 Queues

Processes facilitate communications from SITE 2 and SITE 3 to APP3 and APP2 via APP1.

Monitor shows THIRD PARTY to APP3 processing statistics for each process, including queues.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- nMsgQueSize,
- InMsgRate
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
nMsgQueSize is non-zero value and not reducing as expected, or InMsgRate is not increasing as expected.
Receiving process may not be up and/or there are THIRD PARTY delivery issues.
1) Check status of receiving processes
2) Involve Production Support if necessary.



Stats Monitors: to ACTR IPC Connect Stats

Processes facilitate communications from NASDAQ and SIAC SIPs to Matching Engines and Market Data Database Loaders via CHX Market Data Processors.

Monitor shows connection status between CHX (MDP) applications and Instrument System Activity Reader.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all MDP or ACTR processes are displayed as expected. MDP or ACTR Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check ACTR xml data files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to SITE 2 App Connect Stats

Processes facilitate quote and last sale delivery from SITE 1 to SITE 2 and SITE 3.

Monitor shows connection status between SITE 1 and SITE 2 / SITE 3 as well as processing statistics.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- ConnStat,
- QuoteQue,
- OutMsgRate,
- OutMsgs

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Connstat is not Connected. SITE 1 is not connected to SITE 2.
1) Use Control Utility APP1 Service Controls to control connections.
2) Call Production Control if needed.
QuoteQue is non-zero value and not decreasing as expected, or OutMsgRate or OutMsgs values are not reflecting changes as expected. We may not be sending quotes and/or lastsales as expected.
1) Check process log and/or data files to confirm inbound messages match outbound messages.
2) Work with SITE 2 and Technical Services if necessary.
Stats Monitors:
APP1 to SITE 2 App Connect Stats
(Binary)

Processes facilitate MESSAGES from SITE  1 to SITE 2 and SITE 3.

Monitor shows connection status between SITE 1 and SITE 2 / SITE 3 as well as processing statistics.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- soup status,
- status,
- is ready to send,
- out rate,
- total sent


Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Soup status is not Connected, Status is not ready, or Is ready to send value is not Y.
SITE 1 is not connected to SITE 2.
1) Use Control Utility APP1 Service Controls to control connections.
2) Call Production Control if needed.
Out Rate or Total Sent values are not reflecting changes as expected. We may not be sending MESSAGES as expected.
1) Check process log and/or data files to confirm inbound messages match outbound messages.
2) Work with SITE 2 and Technical Services if necessary.
Stats Monitors:
APP1 To APP2 Queues Stats

Processes facilitate MESSAGE delivery from SITE 1 to SITE 2 and SITE 3.

Monitor shows IPC connection status between APP1 process and other SITE 1 APPS, or THIRD PARTY COMMUNICATION status by topic.

To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or IPC connected processes are displayed as expected. Not all IPC connected services have been started or havn't processed any messages since monitor has been started.
1) Check status of services.
2) Check relevant service log files.
IPC Connected status is not CONNECTED.

Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
THIRD PARTY  connected status is Inactive THIRD PARTY communications has been disabled between the services involved.
1) Check status of process
2) If process is up, call Production Support. 
IPC Connected Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
THIRD PARTY connected Write Queue is non-zero values and not reducing as expected.
THIRD PARTY communications has been disabled between the services involved.
1) Check status of process
2) If process is up, call Production Support. 
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate MESSAGE delivery from SITE 1 to SITE 2 and SITE 3.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 to APP2 App Queues Stats

Processes facilitate MESSAGE delivery from APP1 processes to SITE 2 .

Monitor shows connection status between APP1 process and APP2, and processing statistics.

To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. Not all APP1 services have been started or havn't processed any messages since monitor has been started.
1) Check status of services.
2) Check relevant service log files.
Status is not CONNECTED.

Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
IPC Connected Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate MESSAGE delivery from APP1 processes to APP2.

Monitor shows processing statistics between APP1, APP2 and APP3 processes.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- InMsgs,
- OutMsgs
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
InMsgs values are not equal to or greater than OutMsgs values.

By design, APP1 sends all MESSAGES to APP2 process but APP2 only forwards non-test messages to APP3 for PROCESSING.
We may not be processing as expected.
1) Check APP1 log files to confirm inbound and outbound messages.
2) Work with Production Support if necessary.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate MESSAGE delivery from APP1 processes to APP2.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate MESSAGE delivery from SITE 1 processes to SITE 2.

Monitor shows connection status between APP1 and SITE 2 as well as processing statistics.
To Start:
Use option from PROD MENU:
Market Data Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- OutMsgs,
- InMsgs,
- OutTrdCapRpts
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
OutTrdCapRpts value does not match OutCnt value in APP1  to APP2 Queues monitor.


We may not be processing as expected.
1) Check APP1 log files and APP2 message files to confirm inbound messages match outbound messages.
2) Work with Production Support if necessary.
Stats Monitors:
APP1 Stats

Processes facilitate MESSAGES from SITE 2 utilizing APP1 files.

Monitor shows connection status between SITE 1 and SITE 2 as well as processing statistics.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Connection Name,
- Connection Status,
- Retrans Requests,
- Retrans Messages Sent,
- Connection IP,
- Connection Port


Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Connection status is not LoggedIn.
Subscriber is not logged in and is unable to request or receive MESSAGES.
1) Work with SITE 2 to reconnect, noting that a stop/restart of the APP1 process on our side will affect every subscriber loggedin at the time - and would not be recommended unless absolutely necessary.
Retrans Requests Accepted and Sent values are unexpectedly high. May indicate that there are MESSAGE delivery issues; Especially if seen for several different subscribers.
1) Call Production Support if necessary.
Stats Monitors:
APP1 App Connect Stats

Processes facilitate communications from APP1 and APP2 via APP3.

Monitor shows connection status between APP3 and SITE 2, and processing statistics.

To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open Site 2 is not connected.
1) Use Control Utility APP1 Service Controls to Open Channels.
2) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
SITE 2 may not be processing as expected.
1) Call SITE 2 and work with Technical Services if necessary.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate communications from APP1 to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero when messages are processed. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from APP1 servers to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel connectivity status to APP1 processes.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from APP1 servers to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel connectivity status to APP3 processes.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from APP1 servers to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel connectivity status to APP2 / APP3 processes.  The only messages going through these channels are messages.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1  or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from APP1 servers to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel connectivity status to OMS processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from APP1 servers to SITE 2, SITE 3, SITE 4, APP2 and APP3.

Monitor shows IPC channel connectivity status to APP2 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 App Connect Stats

Processes generate MESSAGES to APP1 via APP2 or APP3 processes and report communication failures from APP1 and APP2 processes.

Monitor shows connection status between APP1 and APP2, APP3 and APP4 processes.

To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Status,
- Write Queue
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is Disconnected or Open APP1, APP2 or APP3 is not connected.
1) Use Control Utility APP1 or APP2 Service Controls to Open Channels.
2) Stop/Restart APP1 or APP2 process if commonality between disconnected statuses can be detected.
3) Call Production Control if needed.
Write Queue is non-zero values and not reducing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 App Processing Stats

Processes generate MESSAGES to APP1 via APP2 or APP3 processes and report communication failures from APP1 and APP2 processes.

Monitor shows testing status and processing statistics of APP1 processes.

To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- ME Names,
- Status,
- Quote Mode,
- Test Result,
- TestDesp
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Status is not Testing APP1  testing has not been enabled.
1) Use Control Utility APP1 Service Controls to control testing.
2) Stop/Restart APP1 process if process isn't responding as expected.
3) Call Production Control if needed.
Quote Mode is Manual
APP1  test cycles have failed 3 consecutive times and not succeeded in 3 subsequent consecutive times.
1) If failure reasons are MESSAGE related, check APP2 processes and processing.
2) If failures are MESSAGE related, check APP2 processing via statistics monitors and APP1 MESSAGE queries if necessary.
3) Call Production Control if needed.
Test Results are not PASS and/or TestDesp are not succesful. APP1 test cycles have failed 3 consecutive times and not succeeded in 3 subsequent consecutive times.
1) If failure reasons are MESSAGE related, check APP3 processes and processing.
2) If failures are MESSAGE related, check APP2 processing via statistics monitors and APP1 queries if necessary.
3) Call Production Control if needed.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate MESSAGE research and modification capabilities with APP2.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero when messages are processed. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 Statistics

Processes facilitate MESSAGE processing.

Monitor shows APP1 status of MESSAGE and processing statistics.

To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- in rate,
- routing enable,
- ors directed connected
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
In_rate and/or msg_in value is zero. APP1 may not be processing as expected.
1) Check APP1 status.
2) Call Production Support if needed.
Routing_enable and/or Ors_dreicted_connected flags are N.
APP1 is not enabled for MESSAGING.
1) Check status of APP2 process.  MESSAGING will be disabled if process is not connected.
2) Use Control Utility APP1 Service Controls to Enable MESSAGING.
3)  Call Production Support if needed.
Stats Monitors:
APP1 Thread Statistics

Processes facilitate MESSAGE processing.

Monitor shows APP1 status and processing statistics.

To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- data queue,
- ctrl que
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Data_queue and/or Ctrl_que column is non-zero value and not decreasing as expected. APP1 may not be processing as expected.
1) Check APP1 status.
2) APP1 may need to be stopped/restarted.  Call Production Support if needed.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate MESSAGE processing.

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero when messages are processed. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate MESSAGE processing.

Monitor shows IPC channel connectivity status between APP2 and APP3 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP2 or APP3 processes are displayed as expected. APP2 or APP3 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP2 or APP3 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 Stats

Processes facilitate messages sent to APP2 to be reported to APP3.

Monitor shows IPC channel processing statistics between OCS service and all LogServers.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service Name,
- Conn Name,
- Status,
- Incnt
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started.
1) Check status of APP1 and APP2 services.
InCnt values are zero when messages are processed. No messages have been sent/received since that monitor has been started.
1) Check APP1 and APP2 files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 IPC Instance Stats

Processes facilitate MESSAGES from SITE 2 that use APP1 services for special routing (to APP2, APP3 or APP4).

Monitor shows IPC channel processing statistics.
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Service,
- Hostname,
- Msgs In,
- Msgs Out
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 processes are displayed as expected. APP1 Service has not been started.
1) Check status of service.
Msgs In and/or Msgs Out are zero when messages are processed. No messages have been sent/received since that monitor has been started.
1) Check APP1 log files.
2) Call Production Support if necessary.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 that use APP1 services for special routing (to APP2, APP3 or APP4).

Monitor shows IPC channel connectivity status to APP1 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate MESSAGES from SITE 2 that use APP1 services for special routing (to APP2, APP3 or APP4).

Monitor shows IPC channel connectivity status to APP2 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 that use APP1 services for special routing (to APP2, APP3 or APP4).

Monitor shows IPC channel connectivity status to APP1 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP1 or APP2 processes are displayed as expected. APP1 or APP2 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1 or APP2 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.
Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:

APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 that use APP1 services for special routing (to APP2, APP3, APP4 and APP5).

Monitor shows IPC channel connectivity status to APP2 / APP3 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP2, APP3 or APP4 processes are displayed as expected. APP1, APP2 or APP3 has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP1, APP2 or APP3 files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.

NOTE: APP2 values will always be zero.  Only the "bridge-to-ME" channels will show non-zero values.

Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
APP1 to APP2 IPC Connect Queues

Processes facilitate communications from SITE 2 that use APP1 services for special routing (to APP2, APP3 or APP4).

Monitor shows IPC channel connectivity status to APP2 processes.  
To Start:
Use option from PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window
- Color of data in columns
- Source,
- Dest,
- Status,
- Queue Size,
- Msgs Out/Sec
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Not all APP2 or APP3 processes are displayed as expected. APP2 or APP3 Service has not been started or hasn't processed any messages since monitor has been started.
1) Check status of service.
2) Check APP2 or APP3 log files.
Status is not CONNECTED. Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process if other processes connecting to the same are showing similar issues; Otherwise, stop/restart source process.
2) Notify Production Support if issues.
Queue size is non-zero value and not decreasing as expected.

NOTE: Queue size will be one until more than one message is generated.  APP2 messaging from APP1 is mostly defunct as of the last update of this documentation.

Messages cannot be sent from source to destination unless IPC channel is connected.
1) Stop/Restart destination process so as not to accidentally delete queued messages; Do not stop/restart source process.
2) Notify Production Support if issues.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate communications from APP1 (via APP2 and APP3 processes) to SITE 2 Destinations.

Monitor shows THIRD PARTY statistics (by topic) for APP2 processes, receiving from APP3 processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Rate and/or MsgCount values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
By matching topic_name, Rcv MsgCount in this monitor does not match total Src MsgCount values in THIRD PARTY moniitor.

SITE 2 may not be receiving messages as expected.
1) Check APP1 files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate communications from APP1 (via APP2 and APP3 processes) to SITE 2.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 processes, receiving from APP2 processes.

SENDER=APP1=Messages sent in FORMAT1.
SENDER=APP2/3=Message sent in FORMAT2.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Msgs_rcved values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check THIRD PARTY message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY Stats

Processes facilitate communications from SITE 2 directly to SITE 1.

Monitor shows THIRD PARTY statistics (by topic) for APP1 processes, receiving from APP2 processes, and sending to APP2.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Statistics are not shown for APP1 process as expected. Process may not be up, or THIRD PARTY Stats have not yet been enabled for process.
1) Check status of process
2) Via Control Utility APP1 Service Controls, enable THIRD PARTY stats for process.
Rate and/or MsgCount values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check THIRD PARTY APP1 files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 RCVRM Stats

Processes facilitate communications from SITE 2 directly to SITE 1.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 processes, receiving from APP2 processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Statistics are not shown for APP1 process as expected. Process may not be up, or THIRD PARTY Stats have not yet been enabled for process.
1) Check status of process
2) Via Control Utility APP1 Service Controls, enable THIRD PARTY stats for process.
Msgs_rcved values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check THIRD PARTY APP1 files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
SITE 2 may not be receiving messages as expected.
1) Check THIRD PARTY APP1 files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate MESSAGE processing.

Monitor shows THIRD PARTY statistics (by topic) for APP1 processes, receiving from APP2, APP3, APP4 and APP5 processes. 

Sending to APP1, APP2 (download messages only), APP3, APP4, APP5, APP6, and APP7(for database loader and APP8) processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Statistics are not shown for APP1 process as expected. Process may not be up, or THIRD PARTY Stats have not yet been enabled for process.
1) Check status of process
2) Via Control Utility APP1 Service Controls, enable THIRD PARTY stats for process.
Rate and/or MsgCount values are not incrementing as expected.
MESSAGE related processing may not be working as expected.
1) Check THIRD PARTY APP1 log files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 RCVRM Stats

Processes facilitate MESSAGE processing.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 processes, receiving from APP2, APP3, APP4 and APP5 processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Statistics are not shown for APP1 process as expected. Process may not be up, or THIRD PARTY Stats have not yet been enabled for process.
1) Check status of process
2) Via Control Utility APP1 Service Controls, enable THIRD PARTY stats for process.
Msgs_rcved values are not incrementing as expected.
MESSAGE related processing may not be working as expected.
1) Check THIRD PARTY APP1 log files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
MESSAGE related processing may not be working as expected.
1) Check THIRD PARTY APP1 log files to confirm THIRD PARTY related processing.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate Application messages sent to APP1 to be reported to APP2.

Monitor shows THIRD PARTY statistics (by topic) for APP1 processes, sending to APP2 processes.  No THIRD PARTY messages are received by APP1. 
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Rate and/or MsgCount values are not incrementing as expected.
Operations may not be seeing all application messages as expected.
1) Check APP1 Binary data files to confirm APP1 related messages.
2) Call Production Support if necessary.
Rate and/or MsgCount values are not incrementing as expected.
Operations may not be seeing all application messages as expected.
1) Check APP1 Binary data files to confirm APP1 related messages.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate Application messages sent to APP1 to be reported to APP2.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 processes. 

Monitor should always be blank as no messages are sent to APP1 using THIRD PARTY protocol.  All messages are sent from APP2 via IPC protocols.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is seen. Must be a configuration problem somewhere.
1) Check processes involved.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 and APP2 Stats

Processes facilitate communications from APP1 to SITE 2 (via APP2 / APP3 processes).

Monitor shows THIRD PARTY statistics (by topic) for APP2 and APP3 processes, receiving from APP1 .  Sending to APP4 processes. 
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Rate and/or MsgCount values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
By matching topic_name, Rcv MsgCount in this monitor does not match total Src MsgCount values in APP1 Stats moniitor.

SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 and APP2 Stats

Processes facilitate communications from APP1 to SITE 2 (via APP2 /APP3 processes).

Monitor shows THIRD PARTY statistics (by topic) for APP1 and APP2 processes, receiving from APP3.  
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Msgs_rcved values are not incrementing as expected.
SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
SITE 2 may not be receiving messages as expected.
1) Check APP1 message files to confirm inbound messages and outbound messages.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate inbound quote processing and database loading.

Monitor shows THIRD PARTY statistics (by topic) for APP1 (recieving download messages from APP2 and sending to APP3, and APP4 and APP5 database loading processes, receiving from APP1 processes. 

To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Rate and/or MsgCount values are not incrementing as expected.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
By matching topic_name, Rcv MsgCount in this monitor does not match total Src MsgCount values.

MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate inbound MESSAGE processing and database loading.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 (receiving download messages from APP2) and APP3 and APP4 Database Loading  processes, receiving from APP1 processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Msgs_rcved values are not incrementing as expected.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate inbound MESSAGES processing and database loading.

Monitor shows THIRD PARTY statistics (by topic) for APP1 (recieving download messages from APP2 and sending to APP3, and APP4 database loading processes, receiving from APP1 processes. 

To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Type,
- Rate,
- Persistence,
- MsgCount
Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Rate and/or MsgCount values are not incrementing as expected.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
By matching topic_name, Rcv MsgCount in this monitor does not match total Src MsgCount values.

MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
Stats Monitors:
THIRD PARTY APP1 Stats

Processes facilitate inbound MESSAGE processing and database loading.

Monitor shows THIRD PARTY "receiving" statistics (by sender, and topic) for APP1 (receiving download messages from APP2) and APP3 Database Loading  processes, receiving from APP1 processes.
To Start:
Use option from PROD MENU:
THIRD PARTY Monitor Menu

To Exit:
Close Window
- Color of data in columns
- ContextName,
- Service,
- Topic_name,
- Lost-Recovered,
- Lost-Unrecovered-Txm,
- Lost-unrecovered-tmo,
-Msgs_rcved

Data is RED. Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.
Msgs_rcved values are not incrementing as expected.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.
Lost-unrecovered values are non-zero.

NOTE: Lost-recovered values may also indicate problems, but that THIRD PARTY auto-recovered lost messages.
MESSAGING may not be being processed or loaded into database as expected.
1) Check APP1 log files and APP2 Binary data files to confirm inbound messages received and loaded into database.
2) Call Production Support if necessary.