APP15 Recovery (MESSAGING Service Router)

APP15 Purpose:

MESSAGING Service Routers receive MESSAGING related MESSAGING and MESSAGING from the APP02 (TXR) or from Clearing.

MESSAGING Service Routers send MESSAGING related MESSAGING and MESSAGING to appropriate order sending firms/vendors.

MESSAGING Service Routes are configured such that:

- the DC01 instance sends drop copies for DC01 APP02s,

- the DC02 instance sends drop copies for DC02 APP02s.

Use the following hyperlinks to jump to the desired section of APP15 documentation:

APP15_Recovery_Considerations

APP15_NTM_Control_Commands

APP15_Troubleshooting_Table

APP15_Monitoring_Considerations

APP15 Recovery Considerations:

Stopping/Restart Processes:

- Use NTM Control Utility - Service Control - Process Controller to stop/restart processes.

- Use APP15 nodes only when moving between nodes. (No real dependencies other than expected processing sites.)

- When stopping/restarting APP15 processes:

1) Notify the MESSAGING firms and vendors to notify them that drop copies will be interrupted.

2) Stop/Restart the APP15 process.

APP15 NTM Control Commands :

- There are no APP15 specific NTM Control Commands.

APP15 Troubleshooting Table:

APP15 Symptom	Impacts	Response
Node Crashes Evidenced by: - In Solarwinds (and outlook), node and processes will be reported down.	MESSAGING firms/vendors will not be able to receive drop copies. Refer to: Server Recovery - Opcon OMS CSI APP15.docx Server Specific Recoveries.	1) Refer to: Server Recovery - Opcon OMS CSI APP15.docx Server Specific Recoveries. 2) Notify Management.

APP15 Monitoring Considerations:

See

Stats Monitors: 29West DCS LBM Stats	To Start:		Key Indicators to Monitor:		Symptom:		Response:
Processes facilitate communications from APP02s (via TXR and APP15 processes) to Order Sending Firm's MESSAGING Destinations. Monitor shows 29 West statistics (by topic) for DCSF processes, receiving from APP15 processes.	PROD MENU: 29 West LBM Monitor Menu To Exit: Close Window		- Color of data in columns - ContextName, - Service, - Topic_name, - Type, - Rate, - Persistence, - MsgCount		Data is RED.		Process is either down or multicast data is not being received by monitor. 1) Check status of process 2) If process is up, call Technical Services.
					Rate and/or MsgCount values are not incrementing as expected.		Firms may not be receiving messages as expected. 1) Check DCS FIX message files to confirm inbound messages and outbound messages. 2) Call Production Support if necessary.
					By matching topic_name, Rcv MsgCount in this monitor does not match total Src MsgCount values in TXR_LBM Stats moniitor.		Firms may not be receiving messages as expected. 1) Check DCS FIX message files to confirm inbound messages and outbound messages. 2) Call Production Support if necessary.
Stats Monitors: 29West DCS LBM RCVRM Stats		To Start:		Key Indicators to Monitor:		Symptom:		Response:
Processes facilitate communications from APP02s (via TXR and APP15 processes) to Order Sending Firm's MESSAGING Destinations. Monitor shows 29 West "receiving" statistics (by sender, and topic) for DCSF processes, receiving from APP15 processes. SENDER=CLEAR=Messages sent in clearing format. SENDER=APP1501/02=Message sent in OSF format.		PROD MENU: 29 West LBM Monitor Menu To Exit: Close Window		- Color of data in columns - ContextName, - Service, - Topic_name, - Lost-Recovered, - Lost-Unrecovered-Txm, - Lost-unrecovered-tmo, -Msgs_rcved		Data is RED.		Process is either down or multicast data is not being received by monitor. 1) Check status of process 2) If process is up, call Technical Services.
						Msgs_rcved values are not incrementing as expected.		Firms may not be receiving messages as expected. 1) Check DCS FIX message files to confirm inbound messages and outbound messages. 2) Call Production Support if necessary.
						Lost-unrecovered values are non-zero. NOTE: Lost-recovered values may also indicate problems, but that 29west auto-recovered lost messages.		Firms may not be receiving messages as expected. 1) Check DCS FIX message files to confirm inbound messages and outbound messages. 2) Call Production Support if necessary.