"Mismatch between metadata in admin service" from Oracle NoSQL Database Community Edition on the Big Data Appliance (Doc ID 1487665.1)

Last updated on MARCH 25, 2014

Applies to:

Oracle NoSQL Database - Version 11.2.1.2.125 and later
Linux x86-64

Symptoms

Goal

Review the CLI verify command verification failure, "Mismatch between metadata in admin service".  If you encounter a problem, such as not being able to start up the Administration Console, verifying the store on each storage node with the CLI runadmin verify command like:
java -jar $KVHOME/lib/kvstore.jar runadmin -port <registry port> -host <hostname>
This could show a "Mismatch between metadata in admin service" verification failure.

 

If the CLI runadmin verify command results in a "Mismatch between metadata in admin service" verification failure, for example like:

$ java -jar $KVHOME/lib/kvstore-1.2.125.jar   runadmin -port 5000 -host node02.example.com
kv-> verify
Verify: starting for BDAKV based upon topology sequence #1008, 1000 partitions and 3 Storage Nodes, version:11gR2.1.2.125
See node03.example.com.com:/lv1/kvroot/BDAKV/log/BDAKV_{0..N}.log for progress messages

Verify: == checking storage node sn1 ==
Verify: Storage Node [sn1] on node02.example.com:5000    Datacenter: BDAKV_DC [dc1]    Status: RUNNING   Ver: 11gR2.1.2.125
Verify:     Admin [admin1]        Status: RUNNING
Verify:     Rep Node [rg1-rn1]    Status: RUNNING,REPLICA at sequence number: 4,598,547 haPort: 5011
Verify: == checking storage node sn2 ==
Verify: Storage Node [sn2] on node01.example.com:5000    Datacenter: BDAKV_DC [dc1]    Status: RUNNING   Ver: 11gR2.1.2.125
Verify: admin2: Mismatch between metadata in admin service and admin2: Param mismatch. Admin metadata= name=adminParams type=adminParams{adminHttpPort=5001, repnodeStartWait=120 SECONDS, collectEnvStats=false, maxTrackedLatency=1000 MILLISECONDS, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, cacheMode=EVICT_LN, activeThreshold=2147483647, latencyCeiling=0, waitTimeout=5 MINUTES, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5011, linkExecWait=200 SECONDS, taskTimeout=5 MINUTES, requestQuiesceTime=60 SECONDS, rnNodeLimitPercent=85, collectorPollPeriod=20 SECONDS, createCSV=false, serviceLogFileLimit=2000000, throughputFloor=0, helperHosts=node02.example.com:5010, serviceStopWait=120 SECONDS, threadDumpInterval=5 MINUTES, adminId=2, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
admin2 metadata= name=adminParams type=adminParams{adminHttpPort=5001, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=true, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, waitTimeout=5 MINUTES, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5011, linkExecWait=200 SECONDS, taskTimeout=5 MINUTES, requestQuiesceTime=60 SECONDS, rnNodeLimitPercent=85, collectorPollPeriod=20 SECONDS, createCSV=false, serviceLogFileLimit=2000000, throughputFloor=0, helperHosts=node02.example.com:5010, serviceStopWait=120 SECONDS, threadDumpInterval=5 MINUTES, adminId=2, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
 diff={collectEnvStats=true}
Verify:     Admin [admin2]        Status: RUNNING
Verify: rg1-rn2: Mismatch between metadata in admin service and rg1-rn2: Param mismatch. Admin metadata= name=rg1-rn2 type=repNodeParams{repNodeId=rg1-rn2, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=false, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5010, linkExecWait=200 SECONDS, rnNodeLimitPercent=85, requestQuiesceTime=60 SECONDS, createCSV=false, collectorPollPeriod=20 SECONDS, throughputFloor=0, serviceLogFileLimit=2000000, serviceStopWait=120 SECONDS, helperHosts=node02.example.com:5011,node03.example.com:5010, threadDumpInterval=5 MINUTES, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, cacheSize=0, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
rg1-rn2 metadata= name=rg1-rn2 type=repNodeParams{repNodeId=rg1-rn2, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=false, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5010, linkExecWait=200 SECONDS, rnNodeLimitPercent=85, requestQuiesceTime=60 SECONDS, createCSV=true, collectorPollPeriod=20 SECONDS, throughputFloor=0, serviceLogFileLimit=2000000, serviceStopWait=120 SECONDS, helperHosts=node02.example.com:5011,node03.example.com:5010, threadDumpInterval=5 MINUTES, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, statsInterval=10 SECONDS, rnRequestThresholdPercent=90, cacheSize=0}
 diff={createCSV=true}
Verify:     Rep Node [rg1-rn2]    Status: RUNNING,REPLICA at sequence number: 4,598,547 haPort: 5010
Verify: == checking storage node sn3 ==
Verify: Storage Node [sn3] on node03.example.com:5000    Datacenter: BDAKV_DC [dc1]    Status: RUNNING   Ver: 11gR2.1.2.125
Verify:     Admin [admin3]        Status: RUNNING
Verify:     Rep Node [rg1-rn3]    Status: RUNNING,MASTER at sequence number: 4,598,547 haPort: 5010

Verification complete, 2 problems found.
Verify: admin2:    Mismatch between metadata in admin service and admin2: Param mismatch. Admin metadata= name=adminParams type=adminParams{adminHttpPort=5001, repnodeStartWait=120 SECONDS, collectEnvStats=false, maxTrackedLatency=1000 MILLISECONDS, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, cacheMode=EVICT_LN, activeThreshold=2147483647, latencyCeiling=0, waitTimeout=5 MINUTES, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5011, linkExecWait=200 SECONDS, taskTimeout=5 MINUTES, requestQuiesceTime=60 SECONDS, rnNodeLimitPercent=85, collectorPollPeriod=20 SECONDS, createCSV=false, serviceLogFileLimit=2000000, throughputFloor=0, helperHosts=node02.example.com:5010, serviceStopWait=120 SECONDS, threadDumpInterval=5 MINUTES, adminId=2, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
admin2 metadata= name=adminParams type=adminParams{adminHttpPort=5001, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=true, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, waitTimeout=5 MINUTES, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5011, linkExecWait=200 SECONDS, taskTimeout=5 MINUTES, requestQuiesceTime=60 SECONDS, rnNodeLimitPercent=85, collectorPollPeriod=20 SECONDS, createCSV=false, serviceLogFileLimit=2000000, throughputFloor=0, helperHosts=node02.example.com:5010, serviceStopWait=120 SECONDS, threadDumpInterval=5 MINUTES, adminId=2, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
 diff={collectEnvStats=true}
Verify: rg1-rn2:    Mismatch between metadata in admin service and rg1-rn2: Param mismatch. Admin metadata= name=rg1-rn2 type=repNodeParams{repNodeId=rg1-rn2, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=false, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5010, linkExecWait=200 SECONDS, rnNodeLimitPercent=85, requestQuiesceTime=60 SECONDS, createCSV=false, collectorPollPeriod=20 SECONDS, throughputFloor=0, serviceLogFileLimit=2000000, serviceStopWait=120 SECONDS, helperHosts=node02.example.com:5011,node03.example.com:5010, threadDumpInterval=5 MINUTES, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, cacheSize=0, rnRequestThresholdPercent=90, statsInterval=10 SECONDS}
rg1-rn2 metadata= name=rg1-rn2 type=repNodeParams{repNodeId=rg1-rn2, repnodeStartWait=120 SECONDS, maxTrackedLatency=1000 MILLISECONDS, collectEnvStats=false, serviceLogFileCount=20, rnMaxActiveRequests=100, adminLogFileCount=20, activeThreshold=2147483647, cacheMode=EVICT_LN, latencyCeiling=0, threadDumpMax=10, storageNodeId=2, adminLogFileLimit=4000000, nodeHostPort=node01.example.com:5010, linkExecWait=200 SECONDS, rnNodeLimitPercent=85, requestQuiesceTime=60 SECONDS, createCSV=true, collectorPollPeriod=20 SECONDS, throughputFloor=0, serviceLogFileLimit=2000000, serviceStopWait=120 SECONDS, helperHosts=node02.example.com:5011,node03.examplecom:5010, threadDumpInterval=5 MINUTES, maxLinkCount=500, eventExpiryAge=30 DAYS, disabled=false, statsInterval=10 SECONDS, rnRequestThresholdPercent=90, cacheSize=0}
 diff={createCSV=true}

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms