E-PRCS: Primary Note for PRCSYSPURGE - The Purge Process (Doc ID 1266338.1)

The quickest and simplest way to ensure the system purge process doesn't run is set the next purge date to a date in the future.

Although this doesn't totally disable the purge, it is a good method to halt the purge process until it is needed again.

To disable the purge:

1) Navigate to PeopleTools - Process Scheduler - System Settings - System Purge Options tab.
2) Set the Next Purge Date value to a future date eg 01/01/2019.
3) Save the record.

In order for changes to take effect, restart all Process Scheduler domains.

All the information available on PurgeProcessRequest(), ArchiveReportMgr(), and PurgeSchdlrServer() functions can be viewed in Application Designer.

In program PRCSYSPURGE the following functions are called:
PurgeProcessRequest();
ArchiveReportMgr();
In program PRCSSRVCLN the following function is called:
PurgeSchdlrServer();

The detailed steps in the code of these PeopleCode functions in PeopleTools 8.4x, are as follows:

PurgeProcessRequest() & PurgeSpecificServer() functions:

Both perform the same tasks except PurgeProcessRequest covers all Servers whereas - PurgeSpecificServer covers a named Server only.

1. Retrieve from the PS_PRCSPURGELIST table for all the run status that needs to be deleted.
2. Delete items marked deleted from the Archive table: PSPRCSRQSTARCH.
3. Delete duplicates in the Archive table: PSPRCSRQSTARCH.
4. Move all single expired processes into the Archive table: PSPRCSRQSTARCH.
5. Move all expired job headers into the Archive table: PSPRCSRQSTARCH.
6. Move all expired job items Into the Archive table: PSPRCSRQSTARCH.
7. Mark items in PSPRCSRQSTARCH for removal.
8. Performs purge of expired processes/jobs from the Process Request tables:

PSPRCSRQST

PSPRCSQUE

PSPRCSCHLDINFO

PSPRCSPARMS

PSPRCSRQSTXFER

PS_PRCSRQSTDIST

PSPRCSRQSTTEXT

PSPRCSRQSTFILE

PSPRCSRQSTMETA

PSPRCSRQSTSTRNG

PSPRCSRQSTTIME

PS_PRCSRQSTNOTIFY

PS_PRCSRQSTURL

PS_PRCSFILE

PSPRCSJOBSTATUS

PSNVSBATCHRSTRT

9. Remove expired entries from message log tables: PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM.

ArchiveReportManager() function:

1. Find reports in the Report Repository marked for Archive.
2. Insert reports into the Archive Table.
3. Inserts reports into the Unpurged Reports table to be Purged.
4. Deletes reports just archived from Auth Table.

PS_CDM_AUTH

5. Delete reports just archived from File List Table.

PS_CDM_FILE_LIST

PS_CDM_LIST

 6. Purges reports from Report Folder tables.

PSRF_RATTR_TBL
PSRF_RSCRTY_TBL
PSRF_RDESC_TBL
PSRF_RDESC_LANG
*PSRF_RINFO_TBL

It is not possible to retrieve the process instance entries in Process Monitor and Report Manager as well as the reports themselves once deleted by the PRCSYSPURGE process.
It is possible, however, to identify the process instance entries and deleted reports through the Archive page in Report Manager.

The following fields for the deleted reports are available for viewing only:

Archive Date:
ReportID:
Prcs Instance:
Report Description:
Request Date/Time:
Output Format:
Process Type:

Navigation:
PeopleTools > Reporting Tools > Report Manager > Archives > Select the date to see the archived list.

Note:
Process Scheduler Administrator rights are required to view this page.
The Report Manager Archives page is used to view reports that have been purged from the report list after they have passed their expiration date.

The parameters for configuring the process scheduler purge are documented in Peoplebooks.

PeopleSoft Process Scheduler > Defining PeopleSoft Process Scheduler Support Information > Defining System Purge Options.

Note:
Any change made under System Purge Options requires the Process Scheduler cache files to be cleared and the scheduler rebooted for changes to take effect.

Use the System Purge Options page to activate the purge process, PRCSYSPURGE, and set purging specifics.

Navigation:
PeopleTools > Process Scheduler > System Settings - System Purge Options Page

Retention Days
Used to calculate the expiration date of reports that are displayed in the Report Manager.
This expiration date is determined by adding the retention days to the date on which the report was generated.

All reports have a EXPIRATION_DATE set at time of report creation.
This can be checked by doing a SQL :
select * from PS_CDM_LIST
or
by going to Report Manager and click "Details" for any report.

This date is set for every report.

The purge deletes rows from the repository tables and reports in the report repository, based on the EXPIRATION_DATE value in PS_CDM_LIST.

If the EXPIRATION_DATE calculated by the Retention Days setting at runtime of the process is older than the day the purge is running, the row will be removed from PS_CDM_LIST and PS_CDM_AUTH, and the report will be removed from the report repository.
When a report is deleted, only select information is written to PS_CDM_LIST_ARCH.
However, this is not a true archive system. The report is not saved anywhere.

The purge process deletes information from the Scheduler tables and the log_output directories of the Process Scheduler servers, based on the Days Before purge setting.
Since the Days Before Purge setting is specific to the Server Definition, it should be set to the same value for all server definitions.
It should also be set to the same value as the Retention Days.
If for some reason there are information in the log_output directories that should have been deleted, it may still be there because there isn't any information in the process scheduler tables concerning that particular process instance.
The only way the purge will know to delete from the log_output directory is if there is information about that directory in the scheduler tables.

The folders in the log_output directory will be removed only if the scheduler tables contain a row for the process instance number.
If the row was removed from the scheduler tables manually, then the folder in log_output will never be touched by the purge process.

Any change made to Retention Days setting will be applicable ONLY to the newly submitted reports after the setting has been changed.

Prior reports will retain their expiration date based off of the old Retention Days setting.
So all existing reports will still expire in the setting that was available at the time of report creation.

Note:
The retention days are based off an expiration date in the tables for each process instance so when the retention days are changed, the Process Scheduler will need to be bounced to pickup the change and then all new processes be assigned the new retention days.

Starting with PeopleTools 8.44, the purge process is executed with an Application Engine program called PRCSYSPURGE.
TLSALL is the process group that is assigned to this Application Engine program.

The purge can be scheduled in 2 ways:
1. Submit the Application Engine program from Process Request page
Navigation:
PeopleTools > Process Scheduler > System Process Requests.
or
2. Schedule the purge to run by configuring the purge settings on the System Purge Options page
Navigation:
PeopleTools > Process Scheduler > System Settings > System Purge Options.

When the purge process is scheduled from the System Purge Options page, the Application Engine, PRCSYSPURGE, will be submitted under the user profile of the ID that is starting the Process Scheduler.
The User Id starting the Process Scheduler must have TLSALL assigned under the Process Profile Permission list in order to execute the PRCSYSPURGE program.

The SQL below will return the Oprid of the Process Scheduler if it has TLSALL assigned.
SELECT C.OPRID FROM PS_PRCSDEFNGRP A, PSAUTHPRCS B, PSOPRCLS C
WHERE A.PRCSTYPE = 'Application Engine' AND A.PRCSNAME = 'PRCSYSPURGE' AND
A.PRCSGRP = B.PRCSGRP AND B.CLASSID = C.OPRCLASS
If the Oprid is not returned as a result of this SQL, the TLSALL process group needs to be assigned to the Process Scheduler Oprid's process profile.

Steps
---------
1. Identify the PeopleSoft Oprid booting the Process Scheduler.
This is value of "UserId= " under [Startup] section from Process Scheduler configuration file, (psprcs.cfg).
2. From the PeopleSoft application, locate this PeopleSoft Oprid's Primary Permission List.
Navigation:
PeopleTools > Security > User Profiles > User Profiles > {Oprid} > General tab > "Primary"
3. Make sure the Primary Permission List has "TLSALL" Process Group assigned to it.
Navigation:
PeopleTools > Security > Permissions & Roles > Permission Lists > {PrimaryPermissionList From#2} > "Process" tab > Click "Process Group Permissions" > Make sure "TLSALL" is listed under "Process Group" column.

The PeopleSoft Oprid that is used to start the Process Scheduler Server must be assigned to the "ProcessSchedulerAdmin" role.

This Oprid should have the role in PeopleTools > Security > User Profiles > User Profiles > {UserID} > Roles tab.

If HTTP is used to transfer reports, it will also be used to delete reports as well from the file system;
to do that means the user account needs authorization to use the "SchedulerTransfer" servlet on the web server.
For that the user account needs the "ProcessSchedulerAdmin" role.

The "PTPT1200" and "PTPT1000" permission lists are required for the Process Scheduler user.

These can be made available to the user by adding the roles "PeopleTools" and "PeopleSoft User".

Check for orphaned rows in the PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM tables that are not found in the PSPRCSRQST and/or PSPRCSQUE tables.

There was an issue with the purge process deleting from all the process request tables except for the PS_MESSAGE_LOG and the PS_MESSAGE_LOGPARM tables.
This behavior has been resolved with PeopleTools 8.44, (Incident ID 598417000).

However, there are many data related scenarios that can cause rows to be left behind by the purge process since the data must fit the appropriate criteria and be found in the correct tables with data for the current environment.

There could be instances where data exists in the PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM tables, but not in the PSPRCSRQST and/or PSPRCSQUE tables.
If this is the case, it indicates that some of the data was orphaned at some time.
The issue of having some process instances in PS_MESSAGE_LOG not in PRCSRQST is reported in Incident ID 1383111000 for PeopleTools 8.46.

Subject:
PS_MESSAGE_LOG & PS_MESSAGE_LOGPARM rows are not being removed by PRCSYSPURGE.

Problem appears to come from a different select of logic used for PS_MESSAGE_LOG & PS_MESSAGE_LOGPARM tables compared to other tables.
Not all rows are being deleted from these 2 tables by this logic.
Once the PRCSINSTANCE is gone from the PSPRCSRQST, it can never be purged because it will never again be inserted into the PSPRCSRQSTARCH table.
The problem is that rows are inserted into the PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM, but not into the PRCSRQST table, these are orphaned and not deleted.
However non-orphaned rows are being purged correctly as delivered.

The workaround provided was to delete these rows manually using a script.

The following 2 SQL statements need to be run.

Delete from PS_MESSAGE_LOG where PROCESS_INSTANCE < (Select min(PRCSINSTANCE) from PSPRCSRQST)
Delete from PS_MESSAGE_LOGPARM where PROCESS_INSTANCE < (Select min (PRCSINSTANCE) from PSPRCSRQST)

Those will purge all rows from the 2 tables PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM for process instance numbers that have already been purged from the other scheduler tables, and will therefore keep all process scheduler tables in sync.
The SQL statements can be put into a script and run it after the regular system purge completes.
Notice that the process instance field name is a bit different in the PS_MESSAGE_LOG and PS_MESSAGE_LOGPARM tables (PROCESS_INSTANCE) versus the process instance field name from PSPRCSRQST table (PRCSINSTANCE).


Starting with PeopleTools 8.49 the purging query has been changed.

First, all process are archived by PSPRCSPURGE if LASTUPDDTTM <= currentDTTM - Retention Days and RUNSTATUS =1 or 10 or 2 or 3 or 4 or 8 or 9.

Subsequently the process is deleted from the process request table.

The PS_MESSAGE_LOG removes all processes that are archived and marked for deletion.

After removing records from PS_MESSAGE_LOG, the MESSAGE_LOGPARM removes all entries which are not in PS_MESSAGE_LOG.

This is working as designed.

But, it is possible, in case an entry is made into PSPRCSRQST table that is not archived, then, the corresponding entry will remain in PS_MESSAGE_LOG forever.

The App Engine purge process PRCSYSPURGE goes to "No Success" in the Process Monitor.

The details of the error are:
Delete Directory Elapsed Time: 0.0630 (65,300)
Application 'PRCSSRVCLN'.'DeleDirs' ended normally (108,4)
The process type Application Engine for the program selected is not valid for server PSNT (65,123)
Process 118388 ABENDED at Step PRCSYSPURGE.SchdlSrv.Step01 (PeopleCode) -- RC = ? (108,524)
Process %s ABENDED at Step %s.%s.%s (Action %s) -- RC = %s
PSAESRV completed service request at 01.05.45 2006-05-08

The PRCSYSPURGE program schedules the Application Engine PRCSRVCLN program if more than one active Process Scheduler servers exist.
PRCSRVCLN cleans up the Log/Output directory for all active servers.
This means that the PRCSYSPURGE is going to run PRCSRVCLN Application Engine program on all Process Scheduler servers.

Hence process type of "Application Engine" needs to be defined in all Process Scheduler servers.
That is, all Process Scheduler servers must be configured to run Application Engine programs.
If not, then this error will occur.