APP03 Recovery (MESSAGING Processes)

APP03 Purpose:

APP03 processes receive order and execution MESSAGING from IBs and send them to External Vendors or Order Management Services.

They then receive related responses from these services and send these back to IBs.

Use the following hyperlinks to jump to the desired section of APP03 documentation:

APP03_Recovery_Considerations

APP03_NTM_Control_Commands

APP03_Troubleshooting_Table

APP03_Monitoring_Considerations

APP03 Recovery Considerations:

Stopping/Restart Processes:

- Use NTM Control Utility - Service Control - Process Controller to stop/restart processes.

- When moving between nodes:

o ALTERNATE NODES are not defined for APP03 services.

o DR NODES must be allocated un-natted nodes.

No node can support more than one natted address at the same time.

- When stopping/restarting APP03 processes:

1) Notify the associated vendor or order management service firm and work in cooperation with them, as appropriate to situations.

2) Notify Tech Services if moving APP03 processes to DR nodes and NAT addresses need to change to accommodate move.

3) Stop the APP03 process.

4) If NOT moving APP03 to new node, skip to step 5.

a) If moving the APP03 to a new node, copy the day’s APP03 FIX engine files to alternate node:

a) Copy: \\data\{APP03}\*.log, *.body, *.header, *.seqnums, *.session file created for the day to the alternate node.

b) Copy: \\data\{APP03}\Global.* file created for the day to the alternate node.

c) If the target folder does not exist on the new node, you must first either create the folder, or copy the entire folder.

d) If these files are not moved before the process restarts on the new node, there will be a chance of sequence number miscommunications between the order sending firm involved and .

5) Start the APP03 process.

6) Open the channel for the process affected and confirm order sending firm connects as expected.

APP03 NTM Control Commands:

Open OSF Channel:

- Use NTM Control Utility – Service Control - APP03 – Open OSF Channel to make OSF connection possible.

Close OSF Channel:

- Use NTM Control Utility – Service Control - APP03 – Close OSF Channel to make OSF connection impossible.

Set Inbound Sequence Number:

- Use NTM Control Utility – Service Control - APP03 – Set Inbound Sequence Number to set Inbound Sequence Number.

Set Outbound Sequence Number:

- Use NTM Control Utility – Service Control - APP03 – Set Outbound Sequence Number to set Outbound Sequence Number.

Enable THIRD PARTY Stats:

- Use NTM Control Utility – Service Control - APP03 – Enable THIRD PARTY Stats to start collection and display of LBM related stats.

Disable THIRD PARTY Stats:

- Use NTM Control Utility – Service Control - APP03 – Disable THIRD PARTY Stats to stop collection and display of LBM related stats.

APP03 Troubleshooting Table:

APP03 Symptom	Impacts	Response
Firm disconnects or Logs out of session Evidenced by: - EMT message saying {firm} is disconnected and/or {firm} is logged out. - Stats monitor shows disconnected in status column. {Firm} will be further identified in EMT message by including “LocalFixId” and “RemoteFixID” as configured in APP03Services.xml file within the disconnect message.	IB is no longer able to send or receive order or drop copy related messages with Vendor or Order Management Service.	1) Contact firm. 2) Work with firm and/or Technical Services as necessary to isolate cause of issues and resolve them. 3) Stop/Restarts of affected application service may help resolve the issue.

APP03 Monitoring Considerations:

Stats Monitors: APP03 App Connect Stats	To Start:	Key Indicators to Monitor:	Symptom:	Response:
Processes facilitate communications from to MESSAGING Destinations. Monitor shows connection status between fix engine and firm.	PROD MENU: Firmswitch Monitoring Menu To Exit: Close Window	- Color of data in columns - Status, - Write Queue	Data is RED.	Process is either down or multicast data is not being received by monitor. 1) Check status of process 2) If process is up, call Technical Services.
			Status is Disconnected or Open	Firm is not connected. 1) Use NTM Control Utility APP03 Service Controls to Open Channels. 2) Call Production Control if needed.
			Write Queue is non-zero values and not reducing as expected.	Firm may not be processing as expected. 1) Call firm and work with Technical Services if necessary.