APP14 Recovery (MESSAGING Service FIX Engine)

APP14 Purpose:

APP14 receives order and execution drop copies from APP01 processes and send them to SITE1 or MESSAGING Services.

They then receive related responses from these services for appropriate error handling.

Use the following hyperlinks to jump to the desired section of APP14 documentation:

APP14_Recovery_Considerations

APP14_NTM_Control_Commands

APP14_Troubleshooting_Table

APP14_Monitoring_Considerations

APP14 Recovery Considerations:

Stopping/Restart Processes:

- Use NTM Control Utility - Service Control - Process Controller to stop/restart processes.

- When moving between nodes:

o ALTERNATE NODES are not defined for APP14 services.

o DR NODES must be allocated un-natted nodes.

No node can support more than one natted address at the same time.

- When stopping/restarting APP03 processes:

1) Notify the associated vendor or MESSAGING service firm and work in cooperation with them, as appropriate to situations.

2) Notify Tech Services if moving APP14 processes to DR nodes and NAT addresses need to change to accommodate move.

3) Stop the APP14 process.

4) If NOT moving APP14 to new node, skip to step 5.

a) If moving the APP14 to a new node, copy the day’s APP03 FIX engine files to alternate node:

a) Copy: \\data\{APP14}\*.log, *.body, *.header, *.seqnums, *.session file created for the day to the alternate node.

b) Copy: \\data\{APP14}\Global.* file created for the day to the alternate node.

c) If the target folder does not exist on the new node, you must first either create the folder, or copy the entire folder.

d) If these files are not moved before the process restarts on the new node, there will be a chance of sequence number miscommunications between the order sending firm involved and .

5) Start the DCSSF process.

6) Open the channel for the process affected and confirm order sending firm connects as expected.

APP14 NTM Control Commands:

Open OSF Channel:

- Use NTM Control Utility – Service Control - APP14 – Open OSF Channel to make OSF connection possible.

Close OSF Channel:

- Use NTM Control Utility – Service Control - APP14 – Close OSF Channel to make OSF connection impossible.

Set Inbound Sequence Number:

- Use NTM Control Utility – Service Control - APP14 – Set Inbound Sequence Number to set Inbound Sequence Number.

Set Outbound Sequence Number:

- Use NTM Control Utility – Service Control - APP14 – Set Outbound Sequence Number to set Outbound Sequence Number.

Enable 29West Stats:

- Use NTM Control Utility – Service Control - APP14 – Enable 29West Stats to start collection and display of LBM related stats.

Disable 29West Stats:

- Use NTM Control Utility – Service Control - APP14 – Disable 29West Stats to stop collection and display of LBM related stats.

DSCF Troubleshooting Table:

APP14 Symptom	Impacts	Response
Firm disconnects or Logs out of session Evidenced by: - EMT message saying {firm} is disconnected and/or {firm} is logged out. - Stats monitor shows disconnected in status column. {Firm} will be further identified in EMT message by including “LocalFixId” and “RemoteFixID” as configured in AMDServices.xml file within the disconnect message.	MESSAGING firm/vendor is no longer able to receive MESSAGING related messages.	1) Contact firm. 2) Work with firm and/or Technical Services as necessary to isolate cause of issues and resolve them. 3) Stop/Restarts of affected application service may help resolve the issue.

APP14 Monitoring Considerations:

Stats Monitors: DCS App Connect Stats	To Start:	Key Indicators to Monitor:	Symptom:	Response:
Processes facilitate communications from (Matching Engines via TXR and APP01 processes) to Order Sending Firm's MESSAGING Destinations. Monitor shows connection status between APP14 and firm as well as processing stats between APP01 (msgin) and APP14 as well as APP14 and firm (msgout).	PROD MENU: Firmswitch Monitoring Menu To Exit: Close Window	- Color of data in columns - fixConnect, - MsgIn, - MsgOut	Data is RED.	Process is either down or multicast data is not being received by monitor. 1) Check status of process 2) If process is up, call Technical Services.
			FixConnect is Disconnected or Open	Firm is not connected. 1) Use NTM Control Utility APP14 Service Controls to Open Channels. 2) Call Production Control if needed.
			MsgIn values are not equal to or greater than MsgOut values. Depending on APP14 configuration, outbound messaging may be filtered such that not all inbound messages received by APP01 will be forwarded to firm.	Firm communications may not be as expected. 1) Check DCS FIX message files to confirm inbound messages and outbound messages. 2) Call firm if necessary.