RDBPROD: RMU/VERIFY shows RMU-W-ABMBITERR: Cause and how to repair (Doc ID 839261.1)

Last updated on JULY 05, 2017

Applies to:

Oracle Rdb Server on OpenVMS - Version 7.2 and later
HP OpenVMS Itanium
HP OpenVMS Alpha


Symptoms

When running an RMU/VERIFY command against an Oracle Rdb database, or specific areas of an Oracle Rdb database, the following messages are returned:

$ RMU/VERIFY test_db
%RMU-W-ABMBITERR, inconsistency between spam page 4361 and bit 5 in area 
bitmap in larea 59 page 2
%RMU-E-BADABMPAG,       error verifying ABM pages
$

The warning indicates inconsistencies between 2 data storage structures:

Each logical area has its own ABM, which in turn indicates which SPAM pages contain entries for that logical area. These structures are explained in the Oracle Rdb7 Guide to Database Maintenance.

Since the range of pages referenced by a SPAM  page may contain a number of logical areas, RMU/VERIFY may report multiple ABMBITERR warnings,  with different ABM pages all pointing to the same SPAM page.

From the warning above, to determine the table or index name referenced by logical area 59 and its physical area:

$ RMU/SHOW AIP/LAREA=59 test_db

Logical area name T
Type: TABLE
Logical area 59 in uniform physical area 2
.
.
.

Examine the ABM on page 2, from the ABMBITERR message, in physical area 2, from the RMU/SHOW AIP.

Dump the ABM page:

$ RMU/DUMP/AREA=2/START=2/END=2 test_db

                   0002 00000002  0000  page 2, physical area 2
                        00000000  0006  checksum = 00000000
               00A8B65F 4E7BBA30  000A  time stamp = 12-MAY-2009 07:54:14.22
                       0000 0004  0012  4 free bytes, 0 locked
 
                        00000003  0016  next area bitmap page 3
                        00000005  001A  max set bit index 5
                        00000000  001E  MBZ '....'
                        00001E60  0022  bitvector count 7776
 
0000000000000000000000000000002F  0026  bitvector '/...............'
00000000000000000000000000000000  0036  bitvector '................'
                                  ::::  (58 duplicate lines)
        000000000000000000000000  03E6  bitvector '............'
 
                        00000000  03F2  MBZ '....'
 
                            803B  03F6  bitmap page for logical area 59
                        00000000  03F8  page sequence number 0
                            0000  03FC  page TSN base 0
                            0000  03FE  MBZ '..'
$

The page tail confirms this is the ABM for logical area 59. The actual bit vector, at address 0026, contains the value 2F. Converting this to binary (to see the bit values) gives 101111. Bit 5 (counting from the right), meaning the fifth SPAM page, is 0. The warning reports that the SPAM at page 4361 should not contain entries for logical area 59 (set to 0 in the ABM) but examining the SPAM page shows that entries do exist. To confirm page 4361 is the fifth SPAM page:

-- page 1 is always the first SPAM
-- the SPAM interval can be found from an RMU/DUMP/HEADER, 1089 in this case, so a SPAM page is followed by 1089 data pages
-- 4361 = 1 + 1089 + 1 + 1089 + 1 + 1089 + 1 +1089 + 1

$ RMU/DUMP/AREA=2/START=4361/END=4361 test_db
 
                   0002 00001109  0000  page 4361, physical area 2 (space
                                        mgmt)
                        22518336  0006  checksum = 22518336
               80000000 00000023  000A  Fast incremental backup TSN = 0:35
                       0000 0001  0012  1 free byte, 0 locked
 
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0016  pages 4362-4425: threshold 3
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0026  pages 4426-4489: threshold 3
.
.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0106  pages 5322-5385: threshold 3
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0116  pages 5386-5449: threshold 3
                              03  0126  page 5450: threshold 3
 
                            016B  0127  363 clumps
                            003B  0129  pages 4362-4364, logical area 59
                            003B  012B  pages 4365-4367, logical area 59
.
.    
                            003B  03FB  pages 5445-5447, logical area 59
                            003B  03FD  pages 5448-5450, logical area 59
 
                              00  03FF  MBZ free '.'

While the ABMBITERR message is a warning, the consequences may be serious. The following possibilities exist:
1. pages 4362 - 5450 contain records that belong to the table
2. pages 4362 - 5450 contained  records that belonged to the table at some previous time, the records have since been deleted
3. table was truncated, when bit 5 was already 0


Case 1:

During a sequential scan, the SPAM pages are used to locate data pages containing rows for the desired table. If the bit indicating a SPAM page is 0 on the ABM, the pages in the interval referenced by that SPAM are not read. To demonstrate:

SQL> SHOW TABLE (COL) t
Information for table T
 
Columns for table T:
Column Name                     Data Type        Domain
-----------                     ---------        ------
CI                              CHAR(7)
I                               INTEGER
C80                             CHAR(80)
 
SQL> SHOW INDEX
User indexes in database with filename test_db
     T_I
SQL> SHOW INDEX t_i
Indexes on table T:
T_I                             with column I
  No Duplicates allowed
  Type is Sorted
  Key suffix compression is DISABLED
  Node size  964
 Store clause:          STORE in TEST_INDEX
 
SQL> SELECT COUNT(*) FROM t ORDER BY i;
 
       50000
1 row selected
SQL> SELECT COUNT(*) FROM t ORDER BY c80;
 
       40199
1 row selected
SQL>

The first count uses the index T_I so does not need to reference any of the ABM or SPAM pages for the table. It is not impacted by bit 5 being 0.

The second count uses a sequential scan through the pages. Data is missed due to the incorrect bit setting. In this case there is a serious problem in the database.

This can be detected by an RMU/VERIFY/DATA/INDEX, which reports a BADIDXREL error as shown below:

$ RMU/VERIFY/DATA/INDEX test_db
%RMU-W-ABMBITERR, inconsistency between spam page 4361 and bit 5 in
area bitmap in larea 59 page 2
%RMU-E-BADABMPAG,       error verifying ABM pages
%RMU-W-BADIDXREL, Index T_I either points to a non-existent record
                  or has multiple pointers to a record in table T.
                  The logical dbkey in the index is 59:4362:0.
%RMU-W-BADIDXREL, Index T_I either points to a non-existent record 
                  or has multiple pointers to a record in table T.
                  The logical dbkey in the index is 59:4362:1.
.
.

 

Case 2:

The most common cause of the ABMBITERR warning is empty pages allocated to a logical area. These pages had data rows for the logical area at one point but those rows were subsequently deleted. The SPAM page still lists those pages.

For this demonstration case, data for table T was initially loaded in the order matching the index T_I. Examining the first data page after SPAM page 4361 shows a row with column CI = 39186 and I = 39186 (hex 9912). If all rows with I > 39000 are deleted, data pages following SPAM page 4361 will be empty.

However, RMU/VERIFY reports the warning since the data pages are still allocated to logical area 59, as indicated on the SPAM page.

To demonstrate, beginning with page 4362 showing a row with the value 39186 for I:

                   0002 0000110A  0000  page 4362, physical area 2
                        C02081E8  0006  checksum = C02081E8
               00A8B65F 4CBD1A40  000A  time stamp = 12-MAY-2009 07:54:11.30
                       0000 000A  0012  10 free bytes, 0 locked
                            0009  0016  9 lines
                       0063 038A  0018  line 0: offset 038A, 99 bytes
                       0063 0326  001C  line 1: offset 0326, 99 bytes
                       0063 02C2  0020  line 2: offset 02C2, 99 bytes
                       0063 025E  0024  line 3: offset 025E, 99 bytes
                       0063 01FA  0028  line 4: offset 01FA, 99 bytes
                       0063 0196  002C  line 5: offset 0196, 99 bytes
                       0063 0132  0030  line 6: offset 0132, 99 bytes
                       0063 00CE  0034  line 7: offset 00CE, 99 bytes
                       0063 006A  0038  line 8: offset 006A, 99 bytes
 
                        00000023  003C  line 0: TSN 35
                        00000023  0040  line 1: TSN 35
                        00000023  0044  line 2: TSN 35
                        00000023  0048  line 3: TSN 35
                        00000023  004C  line 4: TSN 35
                        00000023  0050  line 5: TSN 35
                        00000023  0054  line 6: TSN 35
                        00000023  0058  line 7: TSN 35
                        00000023  005C  line 8: TSN 35
 
            00000000000000000000  0060  free space '..........'
 
                            001F  006A  line 8 (2:4362:8) record type 31
                         00 0001  006C  Control information
                                  ....  94 bytes of static data
20207C00009912202036383139330001  006F   data '..39186  ....|  '
6F6E6D6C6B6A69686766656463626120  007F   data ' abcdefghijklmno'
302F3F3E3C7A79787776757473727170  008F   data 'pqrstuvwxyz<>?/0'
202020202020202020202020207C2020  009F   data '  |             '
20202020202020202020202020202020  00AF   data '                '
    F820202020202020202020202020  00BF   data '             ÃƒÂ¸'
                              00  00CD  padding '.'
.
.
.
$! empty the data pages beyond page 4361
$ MC SQL$
SQL> ATTACH 'FILENAME test_db';
SQL> DELETE FROM t WHERE i > 39000;
11000 rows deleted
SQL> COMMIT;
SQL> EXIT
$
$ RMU/VERIFY/ALL test_db
%RMU-W-ABMBITERR, inconsistency between spam page 4361 and bit 5 in area
 bitmap in larea 59 page 2
%RMU-E-BADABMPAG,       error verifying ABM pages
$
$! check data counts, showing no data is missed
$
$ MS SQL$
SQL> ATTACH 'FILENAME test_db';
SQL> SELECT COUNT(*) FROM t ORDER BY i;
 
       39000
1 row selected
SQL> SELECT COUNT(*) FROM t ORDER BY c80;
 
       39000
1 row selected
SQL> SELECT DBKEY FROM t WHERE i=39000;
                  DBKEY
              59:4341:2
1 row selected
SQL>

The last data row is stored on page 4341. However, a dump of SPAM page 4361 shows pages beyond that are still allocated to logical area 59, as shown below:

                   0002 00001109  0000  page 4361, physical area 2 (space
                                        mgmt)
                        24B78336  0006  checksum = 24B78336
               80000000 0000028C  000A  Fast incremental backup TSN = 0:652
                       0000 0001  0012  1 free byte, 0 locked
 
00000000000000000000000000000000  0016  pages 4362-4425: threshold 0
00000000000000000000000000000000  0026  pages 4426-4489: threshold 0
.
.
00000000000000000000000000000000  0106  pages 5322-5385: threshold 0
00000000000000000000000000000000  0116  pages 5386-5449: threshold 0
                              00  0126  page 5450: threshold 0
 
                            016B  0127  363 clumps
                            003B  0129  pages 4362-4364, logical area 59
                            003B  012B  pages 4365-4367, logical area 59
.
.
                            003B  03FB  pages 5445-5447, logical area 59
                            003B  03FD  pages 5448-5450, logical area 59


In this case, there is no actual problem with the database.

Case 3:

The ABMBITERR warning from Case 1 may lead to further issues if the table is truncated and subsequently backed up and restored. When a table is truncated, the ABM is used to identify the SPAM pages referencing data pages for that logical area. The SPAM pages are updated as shown below.

Original SPAM page:

                   0002 00000001  0000  page 1, physical area 2 (space mgmt)
                        2250722E  0006  checksum = 2250722E
               80000000 00000022  000A  Fast incremental backup TSN = 0:34
                       0000 0001  0012  1 free byte, 0 locked
 
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0016  pages 2-65: threshold 3
.
.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0116  pages 1026-1089: threshold 3
                              03  0126  page 1090: threshold 3
 
                            016B  0127  363 clumps
                            003B  0129  pages 2-4, logical area 59
                            8013  012B  pages 5-7, logical area 59
                            8013  012D  pages 8-10,  logical area 59
.
.
                            003B  03FD  pages 1088-1090, logical area 59
 
                              00  03FF  MBZ free '.'


Following a TRUNCATE:

          ;          0002 00000001  0000  page 1, physical area 2 (space mgmt)
                        35F28492  0006  checksum = 35F28492
               80000000 00000160  000A  Fast incremental backup TSN = 0:352
                       0000 0001  0012  1 free byte, 0 locked
 
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0016  pages 2-65: threshold 3
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0026  pages 66-129: threshold 3
.
.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0116  pages 1026-1089: threshold 3
                              03  0126  page 1090: threshold 3
 
                            016B  0127  363 clumps
                            003B  0129  pages 2-4, logical area 59
                            8013  012B  pages 5-7, deleted by TID 19
                            8013  012D  pages 8-10, deleted by TID 19
.
.
                            8013  03FD  pages 1088-1090, deleted by TID 19
 
                              00  03FF  MBZ free '.'


However, because bit 5 is 0, the fifth SPAM page (page 4361) is not modified.

Before TRUNCATE:

                   0002 00001109  0000  page 4361, physical area 2 (space mgmt)
                        22518336  0006  checksum = 22518336
               80000000 00000023  000A  Fast incremental backup TSN = 0:35
                       0000 0001  0012  1 free byte, 0 locked
 
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0016  pages 4362-4425: threshold 3
.
.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0116  pages 5386-5449: threshold 3
                              03  0126  page 5450: threshold 3
 
                            016B  0127  363 clumps
                            003B  0129  pages 4362-4364, logical area 59
.
.
                            003B  03FD  pages 5448-5450, logical area 59
 
                              00  03FF  MBZ free '.'


After TRUNCATE:

                   0002 00001109  0000  page 4361, physical area 2 (space mgmt)
                        22518336  0006  checksum = 22518336
               80000000 00000023  000A  Fast incremental backup TSN = 0:35
                       0000 0001  0012  1 free byte, 0 locked
 
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0016  pages 4362-4425: threshold 3
.
.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  0116  pages 5386-5449: threshold 3
                              03  0126  page 5450: threshold 3
 
                            016B  0127  363 clumps
                            003B  0129  pages 4362-4364, logical area 59
.
.
                            003B  03FD  pages 5448-5450, logical area 59
 
                              00  03FF  MBZ free '.'


Since RMU/BACKUP looks at SPAM and data pages, the following is stored in the backup file:
- For data pages that have "deleted by TID nn" in the SPAM page, only the page header and the page tail are backed up, no data
- For data pages that have "logical area 0" in the SPAM page, only the page header and the page tail are backed up, no data
- For data pages that have "logical area nn" in the SPAM page, the entire page, including data, is backed up

For the Case 1 database, even though the ABM said SPAM page 4361 did not reference any pages for logical area 59 (bit 5 was 0), data pages referenced on SPAM page 4361 are backed up in their entirety.

If this backup is then restored, these data rows 'reappear'. In addition, because there was an index on the table, and this index contains no pointers to records following  the TRUNCATE, RMU/VERIFY will report DATNOTIDX errors for those records as shown below:

$ RMU/VERIFY/ALL test_db 
%RMU-W-DATNOTIDX, Row in table T is not in any indexes. 
                  Logical dbkey is 59:4362:0. 
%RMU-W-DATNOTIDX, Row in table T is not in any indexes. 
                  Logical dbkey is 59:4362:1.
.
.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms