APP28 Recovery (APPLICATION USE )

 

                APP28 Purpose:

APP28 reads  Application  messages via APP01 processes running on every  application node.

APP28 then broadcasts these messages to the network via multicast to be picked up and displayed by the EMT (Event Messaging Terminal).

 Operations management staff cannot monitor  applications without APP28 working as expected.

 

Use the following hyperlinks to jump to the desired section of APP28 documentation:

APP28_Recovery_Considerations

APP28_NTM_Control_Commands

APP28_Troubleshooting_Table

APP28_Monitoring_Considerations

 

APP28 Recovery Considerations:

                                Stopping/Restart Processes:

-          Use NTM Control Utility - Service Control - Process Controller to stop/restart processes.

-          Use APP28 nodes only when moving between nodes.  (There are no dependencies outside of expected server allocations.)

 

APP28 NTM Control Commands:

There are no APP28 specific NTM Control Commands.

 

APP28 Troubleshooting Table:

APP28 Symptom

Impacts

Response

Node Crashes

 

Evidenced by:

-          In EMT, applications report lost communications to NSPR service.

-          In Solarwinds (and outlook), node and processes will be reported down.

 

-Unless both NSPR services are down, or inaccessible for any reason, there will be no impact.  If both are down,  applications will not be able to start cleanly.

 

Refer to:

Server Recovery - Opcon OMS CSI DCSR.docx

 Server Specific Recoveries.

1)      Refer to:

Server Recovery - Opcon OMS CSI DCSR.docx

 Server Specific Recoveries.

2)      Restart affected FMR processes on alternate nodes.

3)      Notify Management and PTT.

 

 


 

APP28 Monitoring Considerations:

Stats Monitors:
APP28 Stats

To Start:

Key Indicators to Monitor:

Symptom:

Response:

Processes facilitate Application messages sent to APP01 to be reported to EMT monitors.

Monitor shows IPC channel processing statistics between APP28 service and all APP01s.

PROD MENU:
Trading Applications Monitoring Menu

To Exit:
Close Window

- Color of data in columns
- Service Name,
- Conn Name,
- Status,
- Incnt

Data is RED.

Process is either down or multicast data is not being received by monitor.
1) Check status of process
2) If process is up, call Technical Services.

 

 

 

Not all APP28 or APP01 processes are displayed as expected.

APP28 or APP01 Service has not been started.
1) Check status of APP28 and APP01 services.

 

 

 

InCnt values are zero when messages are processed.

No messages have been sent/received since that monitor has been started.
1) Check APP01 and Opcon data files.
2) Call Production Support if necessary.