DBMS_SCHEDULER fails with ORA-27369 and STANDARD_ERROR="Launching external job failed: Invalid username or password" (Doc ID 976049.1)

Last updated on DECEMBER 06, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.1.0.7 to 11.2.0.1 [Release 11.1 to 11.2]
Oracle Solaris on SPARC (64-bit)
Linux x86-64
*** Checked for relevance 25-Sept-2014 ***

Symptoms

On Solaris or on Linux Systems, the scheduler external job using credentials fails with:

ORA-27369: job of type EXECUTABLE failed with exit code: Argument list too long
STANDARD_ERROR="Launching external job failed: Invalid username or password"


EXAMPLE:

1. Create the shell script: test.sh, with the following contents:

#!/bin/ksh
DATE=`date +%Y%m%d`
echo "this is a test " > /testcases/mylog_$DATE.log


2. Make sure that the shell script is of type executable and can run at the OS level:
chmod 777 test.sh
./test.sh
verify that the logfile has been created at the OS level, then remove it for the scheduler test

3. Create the credential, the job and use the credential as an attribute for the job:

exec dbms_scheduler.create_credential('MY_CRED','<os username>','<password>');
begin
dbms_scheduler.create_job(
job_name => 'MY_JOB',
job_type => 'EXECUTABLE',
job_action => '/testcases/test.sh',
auto_drop => FALSE,
enabled => false,
comments => 'Run shell-script');
end;
/
exec dbms_scheduler.Set_Attribute('MY_JOB', 'credential_name', 'MY_CRED' );
exec dbms_scheduler.enable('MY_JOB');
exec dbms_scheduler.run_job('MY_JOB');

4. Check the additional_information column in the scheduler logs. Following error message can be seen:

SQL> select job_name, status , error# , additional_info from dba_scheduler_job_run_details where job_name like '%MY_JOB%';
JOB_NAME
--------------------------------------------------------------------------------
STATUS ERROR#
------------------------------ ----------
ADDITIONAL_INFO
--------------------------------------------------------------------------------
MY_JOB
FAILED 27370
EXTERNAL_LOG_ID="job_75420_2892",
ORA-27369: job of type EXECUTABLE failed with exit code:: Argument list too long
STANDARD_ERROR="Launching external job failed: Invalid username or password"


NOTE: When the credentials attribute is not set for the job, the external job runs successfully.
It only failed when credentials are used.



ANALYSIS:
-------------

1. Oracle binary s-bits and permissions have been checked as described in
<NOTE 961019.1> DBMS_SCHEDULER Extjob Fails With "Login Executable Not Setuid-Root"

2. Traced the $ORACLE_HOME/bin/jssu executable with 'strace' command on OS level.
Note: Jssu is called by the dbms_scheduler when credentials are used.

Trace file has following output:

geteuid() = 0
futex(0x32618030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=66858, ...}) = 0
mmap(NULL, 66858, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b5c35958000
close(3) = 0
open("/lib64/tls/x86_64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/lib64/tls/x86_64", 0x7fff75150f90) = -1 ENOENT (No such file or directory)
open("/lib64/tls/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/lib64/tls", 0x7fff75150f90) = -1 ENOENT (No such file or directory)
open("/lib64/x86_64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/lib64/x86_64", 0x7fff75150f90) = -1 ENOENT (No such file or directory)
open("/lib64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/lib64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/usr/lib64/tls/x86_64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/tls/x86_64", 0x7fff75150f90) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/tls", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/usr/lib64/x86_64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/x86_64", 0x7fff75150f90) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=36864, ...}) = 0
munmap(0x2b5c35958000, 66858) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=66858, ...}) = 0
mmap(NULL, 66858, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b5c35958000
close(3) = 0
open("/lib64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libpam.so", O_RDONLY) = -1 ENOENT (No such file or directory)


This shows that the jssu executable is searching for /usr/lib64/libpam.so which does not exist on the system.

The library file /usr/lib64/libpam.so  is related to PAM which is usually installed on Linux Red Hat systems.

PAM (Pluggable Authentication Modules) is a system security tool which allows system administrators to set authentication policy without having to recompile programs which do authentication.

3. Check the existence of  library file libpam.so
    and check if PAM has been installed on the Linux system with:

  $ rpm -qa | grep pam

This will show you some PAM options when installed, for example like:
pam-0.99.6.2-4.el5
pam-devel-0.99.6.2-4.el5
pam_pkcs11-0.5.3-23
spamassassin-3.2.5-1.el5
pam_smb-1.1.7-7.2.1
pam_passwdqc-1.0.2-1.2.2
pam_passwdqc-1.0.2-1.2.2
pam_krb5-2.2.14-10
pam-0.99.6.2-4.el5
pam_ccreds-3-5
pam_ccreds-3-5

etc...

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms